> ## Documentation Index
> Fetch the complete documentation index at: https://developers.reflection.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Serve with TokenSpeed

> Run Beam on a single GPU server with TokenSpeed

export const TokenSpeedIcon = () => <>
    <img src="/images/logos/tokenspeed.svg" alt="" className="size-6 m-0! shrink-0 dark:hidden" />
    <img src="/images/logos/tokenspeed-dark.svg" alt="" className="size-6 m-0! shrink-0 hidden dark:block" />
  </>;

<Card title="TokenSpeed recipe for Beam" icon={<TokenSpeedIcon />} href="https://lightseek.org/tokenspeed/recipes/models#beam" arrow>
  The full reference for serving Beam with LightSeek's TokenSpeed.
</Card>

This page walks through one setup, for development and evaluation: TokenSpeed in Docker on one server with four B300 GPUs, using the NVFP4 weights and the DFlash2 drafter for speculative decoding.

Before you start, check the requirements and [download the weights](/serving-weights). The commands below use the `MODEL_DIR` that page sets.

<Steps>
  <Step title="Get the image">
    Select a [TokenSpeed image](https://hub.docker.com/r/lightseekorg/tokenspeed) release with Beam and DFlash2 support, including the Beam tool-call and reasoning parsers in its bundled gateway, and pin it by digest:

    ```bash Pull and pin the image icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    export IMAGE='lightseekorg/tokenspeed:beam'
    docker pull "$IMAGE"
    export IMAGE="$(docker image inspect "$IMAGE" \
      --format '{{index .RepoDigests 0}}')"
    ```
  </Step>

  <Step title="Start the server">
    Run on your GPU server:

    ```bash Start the server icon="docker" highlight={7,10,12,17,19} theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    docker run --rm --init --name beam \
      --gpus '"device=0,1,2,3"' --shm-size=64g \
      -p 127.0.0.1:8000:8000 \
      --mount "type=bind,src=$MODEL_DIR,dst=/weights,readonly" \
      -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 \
      --entrypoint tokenspeed "$IMAGE" \
      serve /weights/nvfp4 \
      --host 0.0.0.0 --port 8000 \
      --served-model-name Beam-1.0-501B-A23B \
      --tensor-parallel-size 4 --dtype bfloat16 \
      --quantization nvfp4 --kv-cache-dtype fp8_e4m3 \
      --max-model-len 131072 --max-num-seqs 8 \
      --gpu-memory-utilization 0.9 \
      --chunked-prefill-size 8192 --max-cudagraph-capture-size 8 \
      --attention-backend trtllm_mha --moe-backend flashinfer_trtllm \
      --enable-prefix-caching --disable-kvstore \
      --tool-call-parser beam --reasoning-parser beam \
      --drafter-attention-backend trtllm_mha \
      --speculative-config '{"method":"dflash","model":"/weights/dflash2","num_speculative_tokens":16}'
    ```

    The container reads the models offline from `/weights` and splits Beam across the four GPUs with tensor parallelism. This configuration accepts up to 131,072 tokens of context and eight concurrent requests, parses tool calls and reasoning with the `beam` parsers, and drafts 16 tokens at a time with DFlash2. TokenSpeed's draft width matches the drafter model's `block_size`.
  </Step>

  <Step title="Make a request">
    Once the server finishes loading, check that it lists `Beam-1.0-501B-A23B`:

    ```bash List the served models icon="terminal" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    curl --fail-with-body http://127.0.0.1:8000/v1/models
    ```

    Then send a chat request:

    ```bash Send a chat request icon="terminal" highlight={7-9} theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    curl --fail-with-body http://127.0.0.1:8000/v1/chat/completions \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "Beam-1.0-501B-A23B",
        "messages": [{"role": "user", "content": "Explain tensor parallelism briefly."}],
        "max_tokens": 2048,
        "temperature": 0.7,
        "top_p": 0.99,
        "chat_template_kwargs": {"allow_missing_thinking": false}
      }'
    ```

    The request sets Beam's recommended `temperature` and `top_p`. TokenSpeed doesn't read them from the checkpoint, so send them with every request.

    It also sets `allow_missing_thinking` to `false`, so the request fails if an earlier assistant turn is missing its reasoning, which would otherwise degrade Beam's output. TokenSpeed has no server-wide default for chat template options, so send this with every request too. See [Reasoning across turns](/harmony-format#reasoning-across-turns).
  </Step>

  <Step title="Stop the server">
    ```bash Stop the server icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    docker stop --time 60 beam
    ```
  </Step>
</Steps>

We're grateful to **LightSeek and NVIDIA** for their guidance and support in developing and refining our Beam serving recipes for TokenSpeed.