> ## Documentation Index
> Fetch the complete documentation index at: https://developers.reflection.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# TRL

> Fine-tune Beam with LoRA using TRL's SFTTrainer

[View on GitHub](https://github.com/huggingface/trl/tree/main/examples/sft_beam)
Code in Hugging Face's TRL repository, maintained by Hugging Face.

[TRL](https://github.com/huggingface/trl) is Hugging Face's library for post-training language models. Its `SFTTrainer` runs supervised fine-tuning on Transformers models, with PEFT for LoRA adapters.

TRL's Beam example fine-tunes `Beam-1.0-501B-A23B` with LoRA. It uses expert parallelism for the experts, shards the rest of the model across the GPUs, and trains on the assistant turns of the [Multilingual-Thinking](https://huggingface.co/datasets/HuggingFaceH4/Multilingual-Thinking) dataset, reasoning included.

See [`examples/sft_beam`](https://github.com/huggingface/trl/tree/main/examples/sft_beam) in the TRL repository for the training script, `sft_beam.py`, and the Slurm job that runs it, `sft_beam.slurm`. The script needs TRL 1.16.0 or later and Transformers 5.20.1 or later.