Why MLA and MTP Fight Each Other: Attention Through Arithmetic Intensity
A detailed derivation shows that for single-token decode, the arithmetic intensity of attention depends only on the ratio of query heads to KV heads, with context length and head dims canceling out. MLA's AI is roughly twice the number of query heads, pushing it near the roofline knee on current GPUs, making it unfriendly to MTP.
MLA behaves during decoding like an MQA with head_dims=512+, and has already consumed most of the compute up front, so “MLA+MTP” tends to lose out.