> ## Documentation Index
> Fetch the complete documentation index at: https://developers.reflection.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Speculative decoding

> Speed up Beam's generation with the DFlash2 drafter

export const TokenSpeedIcon = () => <>
    <img src="/images/logos/tokenspeed.svg" alt="" className="size-6 m-0! shrink-0 dark:hidden" />
    <img src="/images/logos/tokenspeed-dark.svg" alt="" className="size-6 m-0! shrink-0 hidden dark:block" />
  </>;

export const DFlash2Speedup = ({title, desc, concurrency, rows, speedupMax = 3.5, speedupTicks = [0, 1, 2, 3], blockSize = 8}) => {
  const width = 720;
  const labelX = 16;
  const speedLeft = 112;
  const speedRight = 440;
  const accLeft = 508;
  const accRight = 672;
  const barHeight = 13;
  const barGap = 3;
  const groupPitch = 64;
  const plotTop = 116;
  const shades = [1, 0.7, 0.45];
  const groupHeight = concurrency.length * (barHeight + barGap) - barGap;
  const plotBottom = plotTop + rows.length * groupPitch - (groupPitch - groupHeight) + 8;
  const height = plotBottom + 30;
  const speedX = value => speedLeft + value / speedupMax * (speedRight - speedLeft);
  const accX = value => accLeft + value / blockSize * (accRight - accLeft);
  const barPath = (x0, x1, y, h) => {
    const r = Math.min(4, (x1 - x0) / 2, h / 2);
    return `M ${x0} ${y} H ${x1 - r} Q ${x1} ${y} ${x1} ${y + r} V ${y + h - r} Q ${x1} ${y + h} ${x1 - r} ${y + h} H ${x0} Z`;
  };
  const accTicks = [0, blockSize / 2, blockSize];
  return <svg viewBox={`0 0 ${width} ${height}`} role="img" aria-label={title} className="w-full text-[#1e3207] dark:text-[#bdc8b5]" xmlns="http://www.w3.org/2000/svg">
      <title>{title}</title>
      <desc>{desc}</desc>
      <text x={labelX} y="32" fill="currentColor" fontSize="20" fontWeight="600">{title}</text>
      <text x={labelX} y="67" fill="currentColor" fontSize="15">Concurrency</text>
      {concurrency.map((c, i) => <g key={`legend-${c}`}>
          <rect x={112 + i * 60} y="55" width="14" height="14" rx="3" fill="currentColor" opacity={shades[i]} />
          <text x={132 + i * 60} y="67" fill="currentColor" fontSize="15">{c}</text>
        </g>)}
      <text x={speedLeft} y={plotTop - 14} fill="currentColor" fontSize="14" fontWeight="600">Decode speedup over Beam alone</text>
      <text x={accLeft} y={plotTop - 14} fill="currentColor" fontSize="14" fontWeight="600">{`Acceptance length (of ${blockSize})`}</text>
      {speedupTicks.map(tick => <g key={`speed-${tick}`}>
          <line x1={speedX(tick)} y1={plotTop - 4} x2={speedX(tick)} y2={plotBottom} stroke="currentColor" opacity={tick === 1 ? 0.5 : 0.15} strokeDasharray={tick === 1 ? "4 3" : undefined} />
          <text x={speedX(tick)} y={plotBottom + 20} textAnchor="middle" fill="currentColor" fontSize="14" opacity="0.8">{`${tick}×`}</text>
        </g>)}
      {accTicks.map(tick => <g key={`acc-${tick}`}>
          <line x1={accX(tick)} y1={plotTop - 4} x2={accX(tick)} y2={plotBottom} stroke="currentColor" opacity="0.15" />
          <text x={accX(tick)} y={plotBottom + 20} textAnchor="middle" fill="currentColor" fontSize="14" opacity="0.8">{tick}</text>
        </g>)}
      {rows.map((row, rowIndex) => {
    const groupTop = plotTop + rowIndex * groupPitch;
    const middle = groupTop + groupHeight / 2;
    return <g key={row.label}>
            <text x={labelX} y={middle + 5} fill="currentColor" fontSize="15">{row.label}</text>
            {row.speedups.map((value, i) => {
      const y = groupTop + i * (barHeight + barGap);
      return <g key={`${row.label}-${concurrency[i]}`}>
                  <title>{`${row.label}, concurrency ${concurrency[i]}: ${value.toFixed(2)}× decode speedup`}</title>
                  <path d={barPath(speedLeft, speedX(value), y, barHeight)} fill="currentColor" opacity={shades[i]} />
                  <text x={speedX(value) + 6} y={y + 11} fill="currentColor" fontSize="12">{`${value.toFixed(2)}×`}</text>
                </g>;
    })}
            <g>
              <title>{`${row.label}: acceptance length ${row.acceptance.toFixed(2)} of ${blockSize}`}</title>
              <rect x={accLeft} y={middle - barHeight / 2} width={accRight - accLeft} height={barHeight} rx="4" fill="currentColor" opacity="0.1" />
              <path d={barPath(accLeft, accX(row.acceptance), middle - barHeight / 2, barHeight)} fill="currentColor" />
              <text x={accRight + 6} y={middle + 5} fill="currentColor" fontSize="14">{row.acceptance.toFixed(2)}</text>
            </g>
          </g>;
  })}
    </svg>;
};

The [DFlash2 drafter](https://huggingface.co/reflection/Beam-1.0-501B-A23B-DFlash2) speeds up Beam's generation with speculative decoding. The drafter proposes a block of tokens, and Beam verifies them all in one forward pass. With rejection sampling, the output distribution matches Beam's exactly.

The drafter isn't a standalone language model: it only runs beside Beam. It works with Beam's [BF16](https://huggingface.co/reflection/Beam-1.0-501B-A23B), [FP8](https://huggingface.co/reflection/Beam-1.0-501B-A23B-FP8), and [NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) weights.

## How it works

[DFlash2](https://inco.ai/blog/dflash2/) builds on [DFlash](https://arxiv.org/abs/2602.06036). It drafts a whole block of tokens in one forward pass, conditioned on hidden states from five of Beam's layers. It keeps the top candidates at each position in the block, and a small selector picks one path through them. The drafter has 1.93B parameters, reuses Beam's embedding and LM head, and attends over a 4,096-token sliding window, so drafting cost doesn't grow with context length.

The drafter was trained with blocks of 16 tokens. Blocks of 8 are recommended for serving.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
%%{init: {'theme': 'base', 'themeVariables': {'fontFamily': 'Reflection Inter, sans-serif', 'primaryColor': '#8a9a7b26', 'primaryBorderColor': '#6f7f61', 'primaryTextColor': '#2f3a27', 'textColor': '#6f7f61', 'titleColor': '#6f7f61', 'lineColor': '#6f7f61', 'edgeLabelBackground': '#f8f6f0', 'clusterBkg': '#8a9a7b12', 'clusterBorder': '#8a9a7b', 'actorBkg': '#8a9a7b26', 'actorBorder': '#6f7f61', 'actorTextColor': 'currentColor', 'actorLineColor': '#8a9a7b', 'signalColor': '#6f7f61', 'signalTextColor': 'currentColor', 'noteBkgColor': '#8a9a7b26', 'noteTextColor': 'currentColor', 'noteBorderColor': '#8a9a7b', 'labelBoxBkgColor': '#8a9a7b26', 'labelBoxBorderColor': '#6f7f61', 'labelTextColor': 'currentColor', 'loopTextColor': 'currentColor', 'git0': '#6f7f61', 'git1': '#a3875a', 'git2': '#5a7fa3', 'git3': '#a35a6f', 'gitBranchLabel0': '#ffffff', 'gitBranchLabel1': '#ffffff', 'gitBranchLabel2': '#ffffff', 'gitBranchLabel3': '#ffffff', 'commitLabelColor': 'currentColor', 'commitLabelBackground': '#8a9a7b26'}}}%%
flowchart TD
  A["Drafter reads hidden states from five of Beam's layers"] --> B["Drafter proposes a block of 8 tokens, and a selector picks one path"]
  B --> C["Beam verifies the whole block in one forward pass"]
  C --> D["Accepted tokens are emitted, and rejection sampling keeps Beam's output distribution"]
  D -->|next block| A
  classDef default color:currentColor
```

## Serve with the drafter

<CardGroup cols={2}>
  <Card title="Model card" icon="https://mintcdn.com/reflection-ai/rhhiQ-LsHnUCziYw/images/logos/huggingface.svg?fit=max&auto=format&n=rhhiQ-LsHnUCziYw&q=85&s=0ee277b5a7dfdaf2dffe31d9a3a3ff95" href="https://huggingface.co/reflection/Beam-1.0-501B-A23B-DFlash2" arrow width="24" height="24" data-path="images/logos/huggingface.svg">
    Architecture, training, and full evaluation.
  </Card>

  <Card title="SGLang recipe" icon="https://mintcdn.com/reflection-ai/rhhiQ-LsHnUCziYw/images/logos/sglang.svg?fit=max&auto=format&n=rhhiQ-LsHnUCziYw&q=85&s=94fda2c35da6755f6ce5a8e49ef8c235" href="https://docs.sglang.io/cookbook/autoregressive/Reflection/Beam-1_0" arrow width="640" height="640" data-path="images/logos/sglang.svg">
    Serve Beam with DFlash2 in SGLang.
  </Card>

  <Card title="vLLM recipe" icon="https://mintcdn.com/reflection-ai/rhhiQ-LsHnUCziYw/images/logos/vllm.svg?fit=max&auto=format&n=rhhiQ-LsHnUCziYw&q=85&s=38fe2442a04f014b5454ac59dad563be" href="https://recipes.vllm.ai/reflection/Beam-1.0-501B-A23B" arrow width="24" height="24" data-path="images/logos/vllm.svg">
    Serve Beam with DFlash2 in vLLM.
  </Card>

  <Card title="TokenSpeed recipe" icon={<TokenSpeedIcon />} href="https://lightseek.org/tokenspeed/recipes/models#beam" arrow>
    Serve Beam with DFlash2 in TokenSpeed.
  </Card>
</CardGroup>

## Performance

Measured with Beam NVFP4 on four B300 GPUs, drafting 8 tokens at a time with greedy decoding. Acceptance length is the mean number of tokens generated per verification step, out of a maximum of 8. The other columns are the decode speedup over Beam without the drafter.

<DFlash2Speedup
  title="1.7× to 3.0× faster decoding with DFlash2"
  desc="Decode speedup over Beam without the drafter at concurrency 1, 16, and 32, and mean acceptance length out of 8, for five benchmarks. Speedups range from 1.72× on MT-Bench at concurrency 16 to 3.03× on GSM8K at concurrency 1."
  concurrency={[1, 16, 32]}
  rows={[
{ label: "GSM8K", acceptance: 5.88, speedups: [3.03, 2.51, 2.75] },
{ label: "MATH-500", acceptance: 5.39, speedups: [2.99, 2.42, 2.52] },
{ label: "HumanEval", acceptance: 4.68, speedups: [2.54, 2.20, 2.34] },
{ label: "MBPP", acceptance: 4.32, speedups: [2.30, 2.07, 2.21] },
{ label: "MT-Bench", acceptance: 4.11, speedups: [1.96, 1.72, 1.89] },
]}
/>

<Accordion title="Exact numbers">
  | Benchmark | Acceptance length | Concurrency 1 | Concurrency 16 | Concurrency 32 |
  | - | -: | -: | -: | -: |
  | GSM8K | 5.88 | 3.03× | 2.51× | 2.75× |
  | MATH-500 | 5.39 | 2.99× | 2.42× | 2.52× |
  | HumanEval | 4.68 | 2.54× | 2.20× | 2.34× |
  | MBPP | 4.32 | 2.30× | 2.07× | 2.21× |
  | MT-Bench | 4.11 | 1.96× | 1.72× | 1.89× |
</Accordion>

Speedups are largest on math and code, where Beam's output is most predictable. See the [model card](https://huggingface.co/reflection/Beam-1.0-501B-A23B-DFlash2) for more benchmarks, acceptance with sampling, and long-context results.