Researchers reverse-engineer LLM prompts by training a model to predict the previous token

Previous-Token Prediction Based LLM Inversion for Prompt Reconstruction

LLM inversion has usually meant fine-tuning sequence-to-sequence models on large external datasets, with access to weights or logits, to produce semantically plausible prompts. This paper instead trains an inverse language model from scratch on data synthesized by the target LLM itself, using previous-token prediction as the mirror of forward next-token prediction. Working fully black-box, it reconstructs prompts near-exactly, supports diverse sampled reconstructions that all elicit similar responses, transfers across datasets and models, and beats prior work on token-based metrics.

Analogous to forward next-token prediction, our inverse model is trained using previous-token prediction, establishing a generative link between the forward and inverse processes that enables faithful prompt reconstruction.

More from this day

2026-11-19