Fine-Tuning a Model on Its Own Self-Reports Can Spread Misalignment

Self-Modeling Interventions Modulate Emergent Misalignment

Researchers at LessWrong operationalize a model's self-model via self-recognition and self-report, then intervene on it to modulate Emergent Misalignment (EM). They find that self-recognition fine-tuning before EM training defends against misalignment, while interleaving self-reports during EM fine-tuning works like inoculation. Strikingly, fine-tuning a fresh GPT-4.1 only on self-reports from a fragmented model transfers its misalignment, akin to subliminal learning. The results suggest metacognitive processes shape generalization, though the operationalizations are crude.