State-of-the-art LLM-based translation models.
ALMA has three generations: ALMA (1st), ALMA-R (2nd), and X-ALMA(3rd NEW!).
ALMA (Advanced Language Model-based TrAnslator) is a many-to-many LLM-based translation model, which adopts a new translation model paradigm: it begins with fine-tuning on monolingual data and is further optimized using high-quality parallel data. This two-step fine-tuning process ensures strong translation performance.
ALMA-R builds upon ALMA models, with further LoRA fine-tuning with our proposed Contrastive Preference Optimization (CPO) as opposed to the Supervised Fine-tuning used in ALMA. CPO fine-tuning requires our triplet preference data for preference learning. ALMA-R now can matches or even exceeds GPT-4 or WMT winners!
X-ALMA (NEW!) extends ALMA(-R) from 6 languages to 50 languages and ensures top-tier performance across 50 diverse languages, regardless of their resource levels. This is achieved by plug-and-play language-specific module architecture and a carefully designed 5-step training recipe with novel Adaptive-Rejection Preference Optimization methods.
Old ALMA Repo:
⭐ Jan. 22 2025 X-ALMA has been accepted at ICLR 2025!
⭐ Oct. 6 2024 X-ALMA is out! Please find the paper here and models & datasets here.
⭐ Jun. 20 2024 We want to give a shout out to SimPO, which shares a similar reference-free preference learning framework with CPO but in a more stable manner due to its special length normalization and target reward margin. The most exciting thing is that CPO and SimPO can potentially be used together! Learn more about CPO-SimPO!
⭐ May.1 CPO paper has been accepted at ICML 2024!
⭐ Mar.22 2024 CPO method now is merged at huggingface trl! See details here.
⭐ Jan.16 2024 ALMA-R is released! Please check more details with our new paper: Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation.
⭐ Jan.16 2024 The ALMA paper: A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models has been accepted at ICLR 2024! Check out more details here!
:star: Supports :star:
We release seven translation models for ALMA series:
Model checkpoints are released at huggingface:
| Models | Base Model Link | LoRA Link |
|---|---|---|
| ALMA-7B (1st gen) | haoranxu/ALMA-7B | - |
| ALMA-7B-LoRA (1st gen) | haoranxu/ALMA-7B-Pretrain | haoranxu/ALMA-7B-Pretrain-LoRA |
| ALMA-7B-R (2nd gen) | haoranxu/ALMA-7B-R (LoRA merged) | - |
| ALMA-13B-LoRA (1st gen) | haoranxu/ALMA-13B | - |
| ALMA-13B-LoRA | haoranxu/ALMA-13B-Pretrain | haoranxu/ALMA-13B-Pretrain-LoRA |
| ALMA-13B-R (2nd gen) | haoranxu/ALMA-13B-R (LoRA merged) | - |
| X-ALMA (NEW, 3rd gen) | X-ALMA Models | - |
Note that ALMA-7B-Pretrain and ALMA-13B-Pretrain are NOT translation models. They only experience stage 1 monolingual fine-tuning (20B tokens for the 7B model and 12B tokens for the 13B model), and should be utilized in conjunction with their LoRA models.
We have also provided the WMT'22 and WMT'23 translation outputs from ALMA-13B-LoRA and ALMA-13B-R in the outputs directory. These outputs also includes our outputs of baselines and can be directly accessed and used for subsequent evaluations.
Datasets used by ALMA and ALMA-R are also released at huggingface now (NEW!)
| Datasets | Train / Validation | Test |
|---|---|---|
| ALMA Human-Written Parallel Data | Parallel train and validation | WMT'22 |
| ALMA-R Triplet Preference Data | Triplet Preference Data | WMT'22 and WMT'23 |
| X-ALMA Data | 50-language parallel data and preference data | WMT'23 and FLORES-200 |
X-ALMA is designed with a plug-and-play architecture, consisting of two components: a base model and language-specific modules, with each module shared across different language groups. There are three ways to load X-ALMA for translation. An example of translating "我爱机器翻译。" into English (X-ALMA should also able to do multilingual open-ended QA).
The first way: loading the merged model where the language-specific module has been merged into the base model (Recommended):
…The second way: loading the base model and language-specific module (Recommended):
model = AutoModelForCausalLM.from_pretrained("haoranxu/X-ALMA-13B-Pretrain", torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, f"haoranxu/X-ALMA-13B-Group{group_id}")
tokenizer = AutoTokenizer.from_pretrained(f"haoranxu/X-ALMA-13B-Group{group_id}", padding_side='left')The third way: loading the base model with all language-specific modules like MoE: (Require large GPU memory)
from modeling_xalma import XALMAForCausalLM
model = XALMAForCausalLM.from_pretrained("haoranxu/X-ALMA", torch_dtype=torch.float16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("haoranxu/X-ALMA", padding_side='left')
# Add `lang="zh"`: specify the language to instruct the model on which group to use for the third loading method during generation.
generated_ids = model.generate(input_ids=input_ids, num_beams=5, max_new_tokens=20, do_sample=True, temperature=0.6, top_p=0.9, lang="zh")The ALMA and ALMA-R translation prompt is:
Translate this from into :
:
:The X-ALMA translation prompt is:
[INST] Translate this from into :
:
: [/INST]conda create -n xalma python=3.11
conda activate xalmaIf you use AMD GPUs, please first install torch with ROCm.
Then install other dependencies:
bash install_alma.shThis is a quick start to evaluate our X-ALMA model. To produce translation outputs for FLORES-200 in both en→cs and cs→en directions, (If you want to evaluate WMT'23 instead, simply pass --override_test_data_path haoranxu/WMT23-Test.), run the following command. Note that You don't need enable --chat_style for ALMA and ALMA-R. This is only for X-ALMA
…The generated outputs will be saved in the your_output_dir. The translation file for the en→cs direction is named test-en-cs, and the file for the cs→en direction is test-cs-en.
The variable ${test_pairs} denotes the translation directions you wish to evaluate. It supports testing multiple directions at once. For example, you can use de-en,en-de,en-cs,cs-en.
Please see more other examples for evaluating ALMA(-R) under the ./evals folder:
Note that this will perform data-parallel evaluation supported by deepspeed: that is, placing a single full copy of your model onto each available GPU and splitting batches across GPUs to evaluate on K GPUs K times faster than on one. For those with limited GPU memory, we offer an alternative method. The user can pass --multi_gpu_one_model to run the process by distributing a single model across multiple GPUs. Please see evaluation examples in evals/alma_13b_r.sh or evals/*no_parallel files.
Here we show how to
Please note that we do not share the training process for X-ALMA specifically, as it would require releasing numerous intermediate checkpoints, making the process overly complex.
To run the CPO fine-tuning with our triplet preference data, run the following command:
bash runs/cpo_ft.sh ${your_output_dir}To execute the OSCAR monolingual fine-tuning, use the following command:
bash runs/mono_ft.sh ${your_output_dir}Once the monolingual data fine-tuning is complete, proceed to the parallel data fine-tuning using the full-weight approach. Execute the following command:
bash runs/parallel_ft.sh ${your_output_dir} $training_pairs$where training_pairs is the translation directions you considered. The default is all 10 directions: de-en,cs-en,is-en,zh-en,ru-en,en-de,en-cs,en-is,en-zh,en-ru.
In Stage 2, there's also an option to employ LoRA for fine-tuning on the parallel data. To do so, execute the following command:
bash runs/parallel_ft_lora.sh ${your_output_dir} $training_pairs$Currently, ALMA supports 10 directions: English↔German, Englishs↔Czech, Englishs↔Icelandic, Englishs↔Chinese, Englishs↔Russian. However, it may surprise us in other directions :)
X-ALMA supports 50 languages and 98 directions (into and from English): da,nl,de,is,no,sv,af,ca,ro,gl,it,pt,es,bg,mk,sr,uk,ru,id,ms,th,vi,mg,fr,hu,el,cs,pl,lt,lv,ka,zh,ja,ko,fi,et,gu,hi,mr,ne,ur,az,kk,ky,tr,uz,ar,he,fa
Our 7B and 13B models are trained on 20B and 12B tokens, respectively. However, as indicated in the paper, fine-tuning 1B tokens should boost the performance substantially. The steps required to fine-tune 1 billion tokens also vary based on your batch size. In our case, the batch size is calculated as follows: 16 GPUs * 4 (batch size per GPU) * 4 (gradient accumulation steps) = 2
No open issues yet, or sync has not completed.