> ## Documentation Index
> Fetch the complete documentation index at: https://developers.reflection.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Scaling out with Dynamo

> Scale Beam across GPUs and nodes with SGLang or vLLM and NVIDIA Dynamo

[NVIDIA Dynamo](https://docs.nvidia.com/dynamo/) is an open-source framework for distributed inference. Use it with SGLang or vLLM to serve Beam across multiple replicas and GPU nodes. Its KV-aware router balances worker load with prompt-cache reuse, and it supports running prefill and decode on separate workers.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
%%{init: {'theme': 'base', 'themeVariables': {'fontFamily': 'Reflection Inter, sans-serif', 'primaryColor': '#8a9a7b26', 'primaryBorderColor': '#6f7f61', 'primaryTextColor': '#2f3a27', 'textColor': '#6f7f61', 'titleColor': '#6f7f61', 'lineColor': '#6f7f61', 'edgeLabelBackground': '#f8f6f0', 'clusterBkg': '#8a9a7b12', 'clusterBorder': '#8a9a7b', 'actorBkg': '#8a9a7b26', 'actorBorder': '#6f7f61', 'actorTextColor': 'currentColor', 'actorLineColor': '#8a9a7b', 'signalColor': '#6f7f61', 'signalTextColor': 'currentColor', 'noteBkgColor': '#8a9a7b26', 'noteTextColor': 'currentColor', 'noteBorderColor': '#8a9a7b', 'labelBoxBkgColor': '#8a9a7b26', 'labelBoxBorderColor': '#6f7f61', 'labelTextColor': 'currentColor', 'loopTextColor': 'currentColor', 'git0': '#6f7f61', 'git1': '#a3875a', 'git2': '#5a7fa3', 'git3': '#a35a6f', 'gitBranchLabel0': '#ffffff', 'gitBranchLabel1': '#ffffff', 'gitBranchLabel2': '#ffffff', 'gitBranchLabel3': '#ffffff', 'commitLabelColor': 'currentColor', 'commitLabelBackground': '#8a9a7b26'}}}%%
flowchart TD
  requests([Requests]) --> frontend["Dynamo frontend<br/>KV-aware router"]
  subgraph nodeA [GPU node]
    w1[SGLang or vLLM worker]
  end
  subgraph nodeB [GPU node]
    w2[SGLang or vLLM worker]
  end
  subgraph pdd [Optional: separate prefill and decode]
    prefill[Prefill workers] --> decode[Decode workers]
  end
  frontend --> w1
  frontend --> w2
  frontend -.-> pdd
  classDef default color:currentColor
```

See NVIDIA's [Beam deployment recipe](https://docs.nvidia.com/dynamo/dev/recipes/beam-1-0-501b) for deployment instructions, or [Serving in production with Kubernetes](/serving-kubernetes) to use the `reflection-serving` Helm chart.

## Pull a maintained image

NVIDIA's NGC images bundle Dynamo with your selected engine. Use the same image for the frontend and workers.

<Tabs>
  <Tab title="SGLang">
    ```bash Pull the image icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    docker pull nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.6.0-beam-1.0-501b-dev.1
    ```
  </Tab>

  <Tab title="vLLM">
    ```bash Pull the image icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    docker pull nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0-beam-1.0-501b-dev.1
    ```
  </Tab>
</Tabs>

## Build from public source

Prerequisites:

* Linux x86-64 build machine
* Docker with BuildKit enabled
* Git
* Python 3 with `venv` support

The commands build each repository's default branch. Use the same Docker daemon for both steps.

<Tabs>
  <Tab title="SGLang">
    <Steps>
      <Step title="Build the engine image">
        SGLang's [upstream Dockerfile](https://github.com/sgl-project/sglang/blob/main/docker/Dockerfile) builds the `runtime` image and includes a packaged version of Dynamo. `BRANCH_TYPE=local` builds the checked-out SGLang source.

        ```bash Build the engine image icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
        git clone https://github.com/sgl-project/sglang.git
        cd sglang
        docker build --platform linux/amd64 \
          -f docker/Dockerfile --target runtime \
          --build-arg BRANCH_TYPE=local \
          --build-arg INSTALL_FLASHINFER_JIT_CACHE=1 \
          -t beam-sglang:local .
        cd ..
        ```
      </Step>

      <Step title="Add Dynamo">
        Build Dynamo from source to replace the packaged version in the engine image. The `RUNTIME_IMAGE` arguments below select `beam-sglang:local` as the runtime base. The rendered Dockerfile builds Dynamo's wheels and installs them into that base, producing `beam-sglang-dynamo:local` for the frontend and workers.

        ```bash Add Dynamo icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
        git clone https://github.com/ai-dynamo/dynamo.git
        cd dynamo
        python3 -m venv .build-venv
        .build-venv/bin/pip install jinja2 pyyaml
        .build-venv/bin/python container/render.py \
          --framework sglang --target runtime \
          --platform linux/amd64 --cuda-version 13.0 \
          --output-short-filename
        docker build --platform linux/amd64 \
          -f container/rendered.Dockerfile \
          --build-arg RUNTIME_IMAGE=beam-sglang \
          --build-arg RUNTIME_IMAGE_TAG=local \
          -t beam-sglang-dynamo:local .
        cd ..
        ```
      </Step>
    </Steps>
  </Tab>

  <Tab title="vLLM">
    <Steps>
      <Step title="Build the engine image">
        vLLM's [upstream Dockerfile](https://github.com/vllm-project/vllm/blob/main/docker/Dockerfile) builds vLLM and its API server. `INSTALL_KV_CONNECTORS=true` adds NIXL dependencies.

        ```bash Build the engine image icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
        git clone https://github.com/vllm-project/vllm.git
        cd vllm
        docker build --platform linux/amd64 \
          -f docker/Dockerfile --target vllm-openai \
          --build-arg INSTALL_KV_CONNECTORS=true \
          -t beam-vllm:local .
        cd ..
        ```
      </Step>

      <Step title="Add Dynamo">
        The engine image above does not include Dynamo. The `RUNTIME_IMAGE` arguments below select `beam-vllm:local` as the runtime base. The rendered Dockerfile builds Dynamo's wheels and installs them into that base, producing `beam-vllm-dynamo:local` for the frontend and workers.

        ```bash Add Dynamo icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
        git clone https://github.com/ai-dynamo/dynamo.git
        cd dynamo
        python3 -m venv .build-venv
        .build-venv/bin/pip install jinja2 pyyaml
        .build-venv/bin/python container/render.py \
          --framework vllm --target runtime \
          --platform linux/amd64 --cuda-version 13.0 \
          --output-short-filename
        docker build --platform linux/amd64 \
          -f container/rendered.Dockerfile \
          --build-arg RUNTIME_IMAGE=beam-vllm \
          --build-arg RUNTIME_IMAGE_TAG=local \
          -t beam-vllm-dynamo:local .
        cd ..
        ```
      </Step>
    </Steps>
  </Tab>
</Tabs>