Extracting Steering Vectors from J Space
The author explores using the Jacobian lens (J space) to derive activation steering vectors for LLMs from just a few concept tokens, bypassing expensive fine-tuning. Testing on Qwen3-1.7B, they find that for simple behaviors like all-caps output, the J-lens-derived vector closely matches a fitted steering vector. For complex behaviors like refusal, the method is brittle and prone to hallucinations, though it shows partial success. The post includes code and detailed experiments.
Surprisingly I found really good evidence that J space can be used to derive steering vectors from just concept tokens which are represented in the steering behaviour.
- a2ff6eeb0
This sounds like a great foundation for an adtech startup.
If you provide free chatbot services, but sell advertisers bids on which steering vectors to use to bias towards products, based on an embedding of the prompt, I bet you'd make a ton of money. For example, Coca Cola would bid on prompts about drinks, and bias towards mentioning Coke products.
I wonder if you could also use a similar method to do product placement in GenAI images and videos, and whether ad revenue would be enough to offset the price of generation. Some ad bids can go pretty high...
- nullbio
- lwarfield
If the author would like, I self computed a j lens for the 27b version of the qwen model. I used it for my own exploration in this area, and can share it if you want.