Serving in production with Kubernetes

Reflection developed this Kubernetes serving solution in partnership with Dell for on-premises deployments. Deploy Beam with the reflection-serving Helm chart. The chart deploys a Dynamo frontend that routes requests to SGLang or vLLM workers, using KV-aware routing. The frontend and every worker read the model from one PVC that you provide.

Prepare the cluster

Prerequisite	Requirement
Kubernetes	Version 1.30 or later
Client tools	Helm 3+ and `kubectl` 1.30+
GPU runtime	NVIDIA drivers, container runtime, and device plugin
Dynamo	Dynamo Operator compatible with the serving image
CPU per worker	32 CPUs requested; 64-CPU limit
Host memory per worker	768 GiB requested; 900 GiB limit
Shared memory per worker	64 GiB for `/dev/shm`, included in the host-memory limit
Model storage	A populated PVC readable by all frontend and worker pods, including across nodes
PDD networking PDD only	RDMA between workers. Replace both `rdma/rdma_shared_device_a` entries in your engine’s PDD file with your cluster’s RDMA resource name.

Get the chart and configuration

Download the Helm chart and configuration files:

Download the chart

curl -fLO https://developers.reflection.ai/files/reflection-serving.zip
unzip reflection-serving.zip
cd serving
export ENGINE_VALUES=sglang-nvfp4.yaml
cp site-values.example.yaml site-values.yaml

The archive unpacks into a serving directory:

serving

reflection-serving

sglang-bf16.yaml

sglang-dflash.yaml

sglang-fp8.yaml

sglang-hicache.yaml

sglang-nvfp4.yaml

sglang-pdd.yaml

site-values.example.yaml

vllm-bf16.yaml

vllm-dflash.yaml

vllm-fp8.yaml

vllm-nvfp4.yaml

vllm-pdd.yaml

reflection-serving is the Helm chart. The YAML files beside it are values files: one engine file per deployment, optional feature files, and your site values. Set ENGINE_VALUES to your configuration:

Model precision	SGLang	vLLM
NVFP4	`sglang-nvfp4.yaml`	`vllm-nvfp4.yaml`
FP8	`sglang-fp8.yaml`	`vllm-fp8.yaml`
BF16	`sglang-bf16.yaml`	`vllm-bf16.yaml`

Edit site-values.yaml for your model storage and GPU nodes:

site-values.yaml

volumes:
  - name: weights
    persistentVolumeClaim:
      claimName: beam-models
      readOnly: true
volumeMounts:
  - name: weights
    mountPath: /weights
    readOnly: true
nodeSelector:
  nvidia.com/gpu.product: NVIDIA-B300-SXM6-AC

Scale out

Add this to site-values.yaml to scale to four replicas:

site-values.yaml

topology:
  replicaCount: 4

Prefill-decode disaggregation

PDD separates prompt processing from token generation. It can help with long prompts or different capacity needs for each stage. Aggregated serving avoids KV transfers and may suit shorter prompts or lower concurrency; the choice is workload dependent. The PDD files add two prefill replicas and one decode replica. Prefill workers transfer the KV cache to decode workers with NIXL over RDMA.

SGLang
vLLM

sglang-pdd.yaml

disaggregation:
  enabled: true
  transferBackend: nixl
  prefill:
    replicaCount: 2
  decode:
    replicaCount: 1
    engineArgs:
      enable-hierarchical-cache: false

vllm-pdd.yaml

disaggregation:
  enabled: true
  transferBackend: nixl
  prefill:
    replicaCount: 2
  decode:
    replicaCount: 1

Install and verify

Install the chart

helm install beam ./reflection-serving -n serving --create-namespace \
  -f "$ENGINE_VALUES" -f site-values.yaml
kubectl get pods -n serving -o wide

Once the frontend and workers are Ready, run the smoke test:

Run the smoke test

helm test beam -n serving --logs --timeout 15m

Production endpoint

Place your TLS and authentication gateway in front of beam-frontend:8000. The chart deploys one frontend; arrange frontend redundancy separately for high availability.

Remove the deployment

helm uninstall beam -n serving
kubectl delete job beam-smoke -n serving --ignore-not-found

Get started

Self-hosting

Third-party providers

Tools

Prepare the cluster

Get the chart and configuration

Scale out

Prefill-decode disaggregation

Install and verify

Production endpoint

Remove the deployment

​Prepare the cluster

​Get the chart and configuration

​Scale out

​Prefill-decode disaggregation

​Install and verify

​Production endpoint

​Remove the deployment

Prepare the cluster

Get the chart and configuration

Scale out

Prefill-decode disaggregation

Install and verify

Production endpoint

Remove the deployment