TRL is Hugging Face’s library for post-training language models. Its SFTTrainer runs supervised fine-tuning on Transformers models, with PEFT for LoRA adapters.
TRL’s Beam example fine-tunes Beam-1.0-501B-A23B with LoRA. It uses expert parallelism for the experts, shards the rest of the model across the GPUs, and trains on the assistant turns of the Multilingual-Thinking dataset, reasoning included.
See examples/sft_beam in the TRL repository for the training script, sft_beam.py, and the Slurm job that runs it, sft_beam.slurm. The script needs TRL 1.16.0 or later and Transformers 5.20.1 or later. Last modified on October 20, 2026
