> ## Documentation Index
> Fetch the complete documentation index at: https://developers.reflection.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Serving in production with Kubernetes

> Deploy Beam on your Kubernetes infrastructure with Helm

Reflection developed this Kubernetes serving solution in partnership with [Dell](https://www.dell.com/en-us/shop/dell-ai-solutions/sc/artificial-intelligence) for on-premises deployments.

Deploy Beam with the `reflection-serving` Helm chart. The chart deploys a Dynamo frontend that routes requests to SGLang or vLLM workers, using KV-aware routing. The frontend and every worker read the model from one PVC that you provide.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
%%{init: {'theme': 'base', 'themeVariables': {'fontFamily': 'Reflection Inter, sans-serif', 'primaryColor': '#8a9a7b26', 'primaryBorderColor': '#6f7f61', 'primaryTextColor': '#2f3a27', 'textColor': '#6f7f61', 'titleColor': '#6f7f61', 'lineColor': '#6f7f61', 'edgeLabelBackground': '#f8f6f0', 'clusterBkg': '#8a9a7b12', 'clusterBorder': '#8a9a7b', 'actorBkg': '#8a9a7b26', 'actorBorder': '#6f7f61', 'actorTextColor': 'currentColor', 'actorLineColor': '#8a9a7b', 'signalColor': '#6f7f61', 'signalTextColor': 'currentColor', 'noteBkgColor': '#8a9a7b26', 'noteTextColor': 'currentColor', 'noteBorderColor': '#8a9a7b', 'labelBoxBkgColor': '#8a9a7b26', 'labelBoxBorderColor': '#6f7f61', 'labelTextColor': 'currentColor', 'loopTextColor': 'currentColor', 'git0': '#6f7f61', 'git1': '#a3875a', 'git2': '#5a7fa3', 'git3': '#a35a6f', 'gitBranchLabel0': '#ffffff', 'gitBranchLabel1': '#ffffff', 'gitBranchLabel2': '#ffffff', 'gitBranchLabel3': '#ffffff', 'commitLabelColor': 'currentColor', 'commitLabelBackground': '#8a9a7b26'}}}%%
flowchart TD
  clients([Clients]) --> gateway[Your TLS and authentication gateway]
  gateway --> frontend["Dynamo frontend<br/>beam-frontend:8000"]
  frontend -->|KV-aware routing| w1[Worker replica 1]
  frontend -->|KV-aware routing| wn[Worker replica N]
  pvc[(Model PVC, read-only)] -.-> frontend
  pvc -.-> w1
  pvc -.-> wn
  classDef default color:currentColor
```

## Prepare the cluster

| Prerequisite | Requirement |
| - | - |
| Kubernetes | Version 1.30 or later |
| Client tools | Helm 3+ and `kubectl` 1.30+ |
| GPU runtime | NVIDIA drivers, container runtime, and device plugin |
| Dynamo | [Dynamo Operator](https://docs.nvidia.com/dynamo/latest/kubernetes/installation/install-dynamo) compatible with the serving image |
| CPU per worker | 32 CPUs requested; 64-CPU limit |
| Host memory per worker | 768 GiB requested; 900 GiB limit |
| Shared memory per worker | 64 GiB for `/dev/shm`, included in the host-memory limit |
| Model storage | A populated PVC readable by all frontend and worker pods, including across nodes |
| PDD networking <Badge color="gray" size="sm">PDD only</Badge> | [RDMA between workers](https://docs.nvidia.com/dynamo/latest/kubernetes/disaggregated-serving/overview). Replace both `rdma/rdma_shared_device_a` entries in your engine's PDD file with your cluster's RDMA resource name. |

## Get the chart and configuration

Download the [Helm chart and configuration files](/files/reflection-serving.zip):

```bash Download the chart icon="terminal" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
curl -fLO https://developers.reflection.ai/files/reflection-serving.zip
unzip reflection-serving.zip
cd serving
export ENGINE_VALUES=sglang-nvfp4.yaml
cp site-values.example.yaml site-values.yaml
```

The archive unpacks into a `serving` directory:

<Tree>
  <Tree.Folder name="serving" defaultOpen>
    <Tree.Folder name="reflection-serving">
      <Tree.Folder name="templates">
        <Tree.File name="_helpers.tpl" />

        <Tree.File name="dynamographdeployment.yaml.tpl" />

        <Tree.File name="smoke-test.yaml.tpl" />
      </Tree.Folder>

      <Tree.File name="Chart.yaml" />

      <Tree.File name="README.md" />

      <Tree.File name="values.schema.json" />

      <Tree.File name="values.yaml" />
    </Tree.Folder>

    <Tree.File name="sglang-bf16.yaml" />

    <Tree.File name="sglang-dflash.yaml" />

    <Tree.File name="sglang-fp8.yaml" />

    <Tree.File name="sglang-hicache.yaml" />

    <Tree.File name="sglang-nvfp4.yaml" />

    <Tree.File name="sglang-pdd.yaml" />

    <Tree.File name="site-values.example.yaml" />

    <Tree.File name="vllm-bf16.yaml" />

    <Tree.File name="vllm-dflash.yaml" />

    <Tree.File name="vllm-fp8.yaml" />

    <Tree.File name="vllm-nvfp4.yaml" />

    <Tree.File name="vllm-pdd.yaml" />
  </Tree.Folder>
</Tree>

`reflection-serving` is the Helm chart. The YAML files beside it are values files: one engine file per deployment, optional feature files, and your site values.

Set `ENGINE_VALUES` to your configuration:

| Model precision | SGLang | vLLM |
| - | - | - |
| NVFP4 | `sglang-nvfp4.yaml` | `vllm-nvfp4.yaml` |
| FP8 | `sglang-fp8.yaml` | `vllm-fp8.yaml` |
| BF16 | `sglang-bf16.yaml` | `vllm-bf16.yaml` |

Edit `site-values.yaml` for your model storage and GPU nodes:

```yaml site-values.yaml icon="file-code" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
volumes:
  - name: weights
    persistentVolumeClaim:
      claimName: beam-models
      readOnly: true
volumeMounts:
  - name: weights
    mountPath: /weights
    readOnly: true
nodeSelector:
  nvidia.com/gpu.product: NVIDIA-B300-SXM6-AC
```

## Scale out

Add this to `site-values.yaml` to scale to four replicas:

```yaml site-values.yaml icon="file-code" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
topology:
  replicaCount: 4
```

## Prefill-decode disaggregation

PDD separates prompt processing from token generation. It can help with long prompts or different capacity needs for each stage. Aggregated serving avoids KV transfers and may suit shorter prompts or lower concurrency; the choice is workload dependent.

The PDD files add two prefill replicas and one decode replica. Prefill workers transfer the KV cache to decode workers with NIXL over RDMA.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
%%{init: {'theme': 'base', 'themeVariables': {'fontFamily': 'Reflection Inter, sans-serif', 'primaryColor': '#8a9a7b26', 'primaryBorderColor': '#6f7f61', 'primaryTextColor': '#2f3a27', 'textColor': '#6f7f61', 'titleColor': '#6f7f61', 'lineColor': '#6f7f61', 'edgeLabelBackground': '#f8f6f0', 'clusterBkg': '#8a9a7b12', 'clusterBorder': '#8a9a7b', 'actorBkg': '#8a9a7b26', 'actorBorder': '#6f7f61', 'actorTextColor': 'currentColor', 'actorLineColor': '#8a9a7b', 'signalColor': '#6f7f61', 'signalTextColor': 'currentColor', 'noteBkgColor': '#8a9a7b26', 'noteTextColor': 'currentColor', 'noteBorderColor': '#8a9a7b', 'labelBoxBkgColor': '#8a9a7b26', 'labelBoxBorderColor': '#6f7f61', 'labelTextColor': 'currentColor', 'loopTextColor': 'currentColor', 'git0': '#6f7f61', 'git1': '#a3875a', 'git2': '#5a7fa3', 'git3': '#a35a6f', 'gitBranchLabel0': '#ffffff', 'gitBranchLabel1': '#ffffff', 'gitBranchLabel2': '#ffffff', 'gitBranchLabel3': '#ffffff', 'commitLabelColor': 'currentColor', 'commitLabelBackground': '#8a9a7b26'}}}%%
flowchart TD
  frontend["Dynamo frontend<br/>beam-frontend:8000"] --> p1[Prefill replica 1]
  frontend --> p2[Prefill replica 2]
  frontend --> d1[Decode replica]
  p1 -->|KV cache, NIXL over RDMA| d1
  p2 -->|KV cache, NIXL over RDMA| d1
  classDef default color:currentColor
```

<Tabs>
  <Tab title="SGLang">
    ```yaml sglang-pdd.yaml icon="file-code" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    disaggregation:
      enabled: true
      transferBackend: nixl
      prefill:
        replicaCount: 2
      decode:
        replicaCount: 1
        engineArgs:
          enable-hierarchical-cache: false
    ```
  </Tab>

  <Tab title="vLLM">
    ```yaml vllm-pdd.yaml icon="file-code" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    disaggregation:
      enabled: true
      transferBackend: nixl
      prefill:
        replicaCount: 2
      decode:
        replicaCount: 1
    ```
  </Tab>
</Tabs>

## Install and verify

```bash Install the chart icon="terminal" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
helm install beam ./reflection-serving -n serving --create-namespace \
  -f "$ENGINE_VALUES" -f site-values.yaml
kubectl get pods -n serving -o wide
```

Once the frontend and workers are Ready, run the smoke test:

```bash Run the smoke test icon="terminal" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
helm test beam -n serving --logs --timeout 15m
```

## Production endpoint

Place your TLS and authentication gateway in front of `beam-frontend:8000`. The chart deploys one frontend; arrange frontend redundancy separately for high availability.

## Remove the deployment

```bash Remove the deployment icon="terminal" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
helm uninstall beam -n serving
kubectl delete job beam-smoke -n serving --ignore-not-found
```