> ## Documentation Index
> Fetch the complete documentation index at: https://developers.reflection.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Serve with vLLM

> Run Beam on a single GPU server with vLLM

<Card title="vLLM recipe for Beam" icon="https://mintcdn.com/reflection-ai/rhhiQ-LsHnUCziYw/images/logos/vllm.svg?fit=max&auto=format&n=rhhiQ-LsHnUCziYw&q=85&s=38fe2442a04f014b5454ac59dad563be" href="https://recipes.vllm.ai/reflection/Beam-1.0-501B-A23B" arrow width="24" height="24" data-path="images/logos/vllm.svg">
  The full reference for serving Beam with vLLM.
</Card>

This page walks through one setup, for development and evaluation: vLLM in Docker on one server with four B300 GPUs, using the NVFP4 weights and the DFlash2 drafter for speculative decoding.

Before you start, check the requirements and [download the weights](/serving-weights). The commands below use the `MODEL_DIR` that page sets.

<Steps>
  <Step title="Get the image">
    Select a [vLLM image](https://docs.vllm.ai/en/latest/deployment/docker/) release with Beam and DFlash support, and pin it by digest:

    ```bash Pull and pin the image icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    export IMAGE='vllm/vllm-openai:reflection-beam'
    docker pull "$IMAGE"
    export IMAGE="$(docker image inspect "$IMAGE" \
      --format '{{index .RepoDigests 0}}')"
    ```
  </Step>

  <Step title="Start the server">
    Run on your GPU server:

    ```bash Start the server icon="docker" highlight={7,10,12,16-19} theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    docker run --rm --init --name beam \
      --gpus '"device=0,1,2,3"' --shm-size=32g \
      -p 127.0.0.1:8000:8000 \
      --mount "type=bind,src=$MODEL_DIR,dst=/weights,readonly" \
      -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 \
      --entrypoint vllm "$IMAGE" \
      serve /weights/nvfp4 \
      --host 0.0.0.0 --port 8000 \
      --served-model-name Beam-1.0-501B-A23B \
      --tensor-parallel-size 4 \
      --dtype bfloat16 \
      --max-model-len 262144 --max-num-seqs 8 \
      --gpu-memory-utilization 0.95 \
      --enable-prefix-caching \
      --enable-auto-tool-choice \
      --tool-call-parser beam --reasoning-parser beam \
      --default-chat-template-kwargs '{"allow_missing_thinking": false}' \
      --structured-outputs-config '{"backend":"xgrammar","reasoning_parser":"beam","disable_any_whitespace":true}' \
      --speculative-config '{"method":"dflash","model":"/weights/dflash2","num_speculative_tokens":8}'
    ```

    The container reads the models offline from `/weights` and splits Beam across the four GPUs with tensor parallelism. This configuration accepts up to 262,144 tokens of context and eight concurrent requests, parses tool calls and reasoning with the `beam` parsers, and drafts eight tokens at a time with DFlash2. Structured outputs use the xgrammar backend.

    `allow_missing_thinking` set to `false` makes a request fail if an earlier assistant turn is missing its reasoning, which would otherwise degrade Beam's output. See [Reasoning across turns](/harmony-format#reasoning-across-turns).
  </Step>

  <Step title="Make a request">
    Once the server finishes loading, check that it lists `Beam-1.0-501B-A23B`:

    ```bash List the served models icon="terminal" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    curl --fail-with-body http://127.0.0.1:8000/v1/models
    ```

    Then send a chat request:

    ```bash Send a chat request icon="terminal" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    curl --fail-with-body http://127.0.0.1:8000/v1/chat/completions \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "Beam-1.0-501B-A23B",
        "messages": [{"role": "user", "content": "Explain tensor parallelism briefly."}],
        "max_tokens": 2048
      }'
    ```

    A request that doesn't set `temperature` or `top_p` uses Beam's recommended `0.7` and `0.99`, which vLLM reads from the checkpoint's `generation_config.json`.
  </Step>

  <Step title="Stop the server">
    ```bash Stop the server icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    docker stop --time 60 beam
    ```
  </Step>
</Steps>

We're grateful to **Inferact and NVIDIA** for their guidance and support in developing and refining our Beam serving recipes for vLLM.