> ## Documentation Index
> Fetch the complete documentation index at: https://developers.reflection.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Serve with SGLang

> Run Beam on a single GPU server with SGLang

<Card title="SGLang recipe for Beam" icon="https://mintcdn.com/reflection-ai/rhhiQ-LsHnUCziYw/images/logos/sglang.svg?fit=max&auto=format&n=rhhiQ-LsHnUCziYw&q=85&s=94fda2c35da6755f6ce5a8e49ef8c235" href="https://docs.sglang.io/cookbook/autoregressive/Reflection/Beam-1_0" arrow width="640" height="640" data-path="images/logos/sglang.svg">
  The full reference for serving Beam with SGLang.
</Card>

This page walks through one setup, for development and evaluation: SGLang in Docker on one server with four B300 GPUs, using the NVFP4 weights and the DFlash2 drafter for speculative decoding.

Before you start, check the requirements and [download the weights](/serving-weights). The commands below use the `MODEL_DIR` that page sets.

<Steps>
  <Step title="Get the image">
    Select an [SGLang image](https://hub.docker.com/r/lmsysorg/sglang) release with Beam and DFlash support, and pin it by digest:

    ```bash Pull and pin the image icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    export IMAGE='lmsysorg/sglang:beam'
    docker pull "$IMAGE"
    export IMAGE="$(docker image inspect "$IMAGE" \
      --format '{{index .RepoDigests 0}}')"
    ```
  </Step>

  <Step title="Start the server">
    Run on your GPU server:

    ```bash Start the server icon="docker" highlight={9-11,18-21,23} theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    docker run --rm --init --name beam \
      --gpus '"device=0,1,2,3"' --shm-size=32g \
      -p 127.0.0.1:8000:8000 \
      --mount "type=bind,src=$MODEL_DIR,dst=/weights,readonly" \
      -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 \
      --entrypoint python3 "$IMAGE" \
      -m sglang.launch_server \
      --host 0.0.0.0 --port 8000 \
      --model-path /weights/nvfp4 --served-model-name Beam-1.0-501B-A23B \
      --tp-size 4 --dtype bfloat16 \
      --context-length 262144 --max-running-requests 8 \
      --mem-fraction-static 0.9 --max-total-tokens 262144 \
      --attention-backend trtllm_mha --moe-runner-backend flashinfer_trtllm \
      --kv-cache-dtype fp8_e4m3 \
      --chunked-prefill-size 8192 --max-prefill-tokens 8192 \
      --cuda-graph-max-bs-decode 8 --page-size 64 \
      --swa-full-tokens-ratio 0.25 \
      --tool-call-parser beam --reasoning-parser beam \
      --default-chat-template-kwargs '{"allow_missing_thinking": false}' \
      --speculative-algorithm DFLASH \
      --speculative-draft-model-path /weights/dflash2 \
      --speculative-draft-load-format safetensors \
      --speculative-num-draft-tokens 8 \
      --speculative-draft-attention-backend trtllm_mha \
      --speculative-draft-kv-cache-dtype fp8_e4m3 \
      --speculative-draft-window-size 4096 \
      --json-model-override-args '{"max_position_embeddings":262144}' \
      --weight-loader-disable-mmap --weight-loader-drop-cache-after-load \
      --model-loader-extra-config '{"enable_multithread_load":false}'
    ```

    The container reads the models offline from `/weights` and splits Beam across the four GPUs with tensor parallelism. This configuration accepts up to 262,144 tokens of context and eight concurrent requests, parses tool calls and reasoning with the `beam` parsers, and drafts eight tokens at a time with DFlash2.

    `allow_missing_thinking` set to `false` makes a request fail if an earlier assistant turn is missing its reasoning, which would otherwise degrade Beam's output. See [Reasoning across turns](/harmony-format#reasoning-across-turns).
  </Step>

  <Step title="Make a request">
    Once the server finishes loading, check that it lists `Beam-1.0-501B-A23B`:

    ```bash List the served models icon="terminal" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    curl --fail-with-body http://127.0.0.1:8000/v1/models
    ```

    Then send a chat request:

    ```bash Send a chat request icon="terminal" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    curl --fail-with-body http://127.0.0.1:8000/v1/chat/completions \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "Beam-1.0-501B-A23B",
        "messages": [{"role": "user", "content": "Explain tensor parallelism briefly."}],
        "max_tokens": 2048
      }'
    ```

    A request that doesn't set `temperature` or `top_p` uses Beam's recommended `0.7` and `0.99`, which SGLang reads from the checkpoint's `generation_config.json`.
  </Step>

  <Step title="Stop the server">
    ```bash Stop the server icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    docker stop --time 60 beam
    ```
  </Step>
</Steps>

We're grateful to **RadixArk and NVIDIA** for their guidance and support in developing and refining our Beam serving recipes for SGLang.