Cache-to-Cache lets LLMs talk in KV-caches, not text, for 2.5x faster inference
Cache-to-Cache: Direct Semantic Communication Between Large Language Models
Multi-LLM systems usually communicate through text, which loses semantic richness and adds token-by-token latency. Cache-to-Cache (C2C) instead projects and fuses the source model's KV-cache directly into the target model's cache, using a learnable gate to pick which layers benefit. It beats individual models by 6.4–14.2% in average accuracy and text-based communication by 3.1–5.4%, while running about 2.5x faster. Published at ICLR'26, with code on GitHub.
Can LLMs communicate beyond text? Oracle experiments show that enriching the KV-Cache semantics can improve response quality without increasing cache size, supporting KV-Cache as an effective medium for inter-model communication.