LensVLM: Selective Context Expansion for Compressed Visual Representation of Text

LensVLM-9B by Apple

Vision-Language Models (VLMs) can process text as rendered images, bypassing tokenization, but accuracy degrades as compression increases because characters shrink below the vision encoder's effective resolution. LensVLM, an inference framework and post-training recipe built on Qwen3.5-9B-Base, addresses this by enabling VLMs to scan compressed images and selectively expand only relevant regions to their uncompressed form via learned tools. It maintains accuracy comparable to the full-text upper bound at 4.3x effective compression and outperforms retrieval-based, text- and visual-compression baselines up to 10.1x effective compression across seven text QA benchmarks. LensVLM also generalizes to multimodal document and code understanding tasks, with gains growing as compression increases. Analysis shows training makes visual compression robust to rendering choices, and as compression grows, the model increasingly relies on expanded content rather than unreliable visual reading. Practical tool-choice guidance emerges: text expansion suits rendered text, while high-resolution image expansion benefits native documents where layout cues are task-relevant.