Requirements
GPUs
Four NVIDIA B300 SXM GPUs connected by NVLink, on a Linux amd64 host.
Docker
Docker with the NVIDIA Container Toolkit and a compatible NVIDIA driver.
Model access
Access to the Beam NVFP4 model and the DFlash2 drafter.
Download the model
1
Install the Hugging Face CLI
Install the Hugging Face CLI, and run
hf auth login if the model requires authentication.2
Choose a download directory
The serving guides mount this directory into the container.
3
Download the model
The SGLang, vLLM, and TokenSpeed guides use NVFP4. Beam is also published in BF16 and FP8.
- NVFP4
- FP8
- BF16
The NVFP4 model is 273 GiB.
Download the NVFP4 model
4
Download the DFlash2 drafter
The serving guides use the DFlash2 drafter for speculative decoding. It works with all three precisions.
Download the DFlash2 drafter