Speculative Decoding on AMD GPUs: vLLM's Mixed Results

Speculative Decoding in vLLM on AMD GPUs

Speculative decoding lets vLLM verify multiple drafted tokens in a single pass of the target model, potentially boosting throughput. But tests on AMD Instinct MI300X and MI355X GPUs with ROCm show the payoff varies widely: it depends on the drafting method (native MTP, Gemma 4 MTP, EAGLE-3, DFlash, DSpark), proposal length, model family, and workload. This post explains how each method works, how to enable them, and what to watch for when tuning.