Jane Street's AI tutor refuses to give you the answer
Jane Street: Socratic Tutor
Jane Street published a Socratic tutor skill for AI agents that teaches by asking questions instead of lecturing. It runs a short diagnostic to place you on a 0-5 skill map per subskill, then drills one question at a time, never handing over answers until you've failed two or three attempts. Levels are earned through repeated checks and unannounced spaced review, and flattery is explicitly banned.
Do not flatter. "Not quite — why would X happen if Y were true?" beats "Great try!".
- Tar90
I've been experimenting with a lot of these learning skills (see also, e.g., https://github.com/mattpocock/skills/blob/main/skills/produc..., which I think addresses the progress tracking part better than this skill) and they're great for getting simple answers, getting unstuck on exercises, getting some intuition through simulations, etc, but I still feel some more structure is needed for effective self-teaching. Has anyone tried using a hard book as a sort of course structure and asking the LLM to teach you through it? You could of course just read the book, but a lot of advanced subjects don't have great learning materials.
- zkmon
This is really cool. I think we will see more of these, inching towards 'refined' thought and expression percolating from AI models into human minds.
I noticed that the quality of email text has gone up around my work these days. everyone is sounding like a master of drafting emails. Ofcourse Ai has a role in it. But real cool thing about this is, this is pulling up people's thinking to have a structure, minimalism and sequencing of expression. Same as how textile industry has enabled people to have an style and expression in clothing.
- pkoird
Is it only me who's thinking what's so special about this and whys everyone oohing and aahing over this? Afaik, people I know (and me) we've been doing this ever since LLMs became capable enough.
- sigbottle
I don't think a simple prompt will exceed any kind of memory system. Nothing fancy, but you need the agent to tune to your quirks; I find that a prompt to try and distill everything a priori is too annoying.
Been just using less of chat recently and using more codex for learning - a direct shared interactable terminal is nice, it's still relatively cheap, and importantly you can log specific thoughts to say a git repo for memory.
It's not attempting to be some kind of knowledge ontology graph like many businesses want - you just want it to understand the specific quirks of your specific learning path, which may not be formalizable or worth be spending time to step back and formalize every little thing.
- Mikhail_Edoshin
Socratic questions are not asked blindly. They are meant to help the student to realize the incompleteness or falsety of what they think they know. That us, to realise they do not actually know something and thus put them into a position of not knowing, which is the only way to gain understanding.
One man said, that enlightement (as in Zen) cannot be faked. Whoever had it, will be able to tell if someone else had it or not. There is no way to merely sound like it. Socratic method is like enlightement.
- genxy
Socratic tutoring is the first thing I did when ChatGPT first shipped. Your liberal arts education is a "huge unlock" if you just apply it.
See also https://en.wikipedia.org/wiki/Mastery_learning
https://en.wikipedia.org/wiki/Programmed_learning
Make your own custom skinner boxes. https://en.wikipedia.org/wiki/Operant_conditioning_chamber
- lemax
I’ve struggled to get AI to teach me things well without spilling hints or giving answers away. Assuming this was built on some solid evals this could be awesome.
> When the user is wrong, do not give the answer. Ask a narrower question that exposes the contradiction in their reasoning
- bsenftner
I've written an entire Socratic AI based office suite with project management and media creation, current count over 1500 agents:
It's comprehensive, it was written at an immigration law firm, so it has all kinds of language and other perspective understanding tools, as well as a complete media suite to aid in the creation of visual explainers, courseware, and marketing media.
https://midombot.com/b1/blog/MI/method-intelligence-presents...