百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
M

mTet

> 编程语言
开源

MTet: 英语和越南语的多域翻译

200 stars0 点赞0 次浏览
访问官网GitHub

工具介绍

MTet: 英语和越南语的多域翻译

MTet: Multi-domain Translation for English-Vietnamese

Release

  • v2.0 (Current)
  • v1.0

Introduction

We are excited to introduce a new larger and better quality Machine Translation dataset, MTet, which stands for Multi-domain Translation for English and VieTnamese. In our new release, we extend our previous dataset (v1.0) by adding more high-quality English-Vietnamese sentence pairs on various domains. In addition, we also show our new larger Transformer models can achieve state-of-the-art results on multiple test sets.

Get data and model at Google Cloud Storage

Visit our paper or blog post for more details.

HuggingFace

…

Results

Citation

…

Using the code

This code is build on top of vietai/dab:

To prepare for training, generate tfrecords from raw text files:

bash
python t2t_datagen.py \
--data_dir=$path_to_folder_contains_vocab_file \
--tmp_dir=$path_to_folder_that_contains_training_data \
--problem=$problem

To train a Transformer model on the generated tfrecords

bash
python t2t_trainer.py \
--data_dir=$path_to_folder_contains_vocab_file_and_tf_records \
--problem=$problem \
--hparams_set=$hparams_set \
--model=transformer \
--output_dir=$path_to_folder_to_save_checkpoints

To run inference on the trained model:

bash
python t2t_decoder.py \
--data_dir=$path_to_folde_contains_vocab_file_and_tf_records \
--problem=$problem \
--hparams_set=$hparams_set \
--model=transformer \
--output_dir=$path_to_folder_contains_checkpoints \
--checkpoint_path=$path_to_checkpoint

In this colab, we demonstrated how to run these three phases in the context of hosting data/model on Google Cloud Storage.

Dataset

Our data contains roughly 4.2 million pairs of texts, ranging across multiple different domains such as medical publications, religious texts, engineering articles, literature, news, and poems. A more detail breakdown of our data is shown in the table below.

v1 v2 (MTet)

Fictional Books 333,189 473,306

Legal Document 1,150,266 1,134,813

Medical Publication 5,861 13,410

Movies Subtitles 250,000 721,174

Software 79,912 79,132

TED Talk 352,652 303,131

Wikipedia 645,326 1,094,248

News 18,449 18,389

Religious texts 124,389 48,927

Educational content 397,008 213,284

No tag 5,517 63,896

Total 3,362,569 4,163,710

Data sources is described in more details here.

Acknowledgment

We would like to thank Google for the support of Cloud credits and TPU quota!

Issues· 5 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

Python

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月18日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言