Automation Without Understanding: The Strategic Error of Losing Math Capacity
I argue that relying on AI for research while weakening the human pipeline to understand it is a critical strategic error. Mathematical capacity is vital infrastructure, not just a byproduct of theorem production. Drawing on the recent AI disproof of an Erdős conjecture, I propose treating this human skill like semiconductor capability and mandating that AI systems expose their reasoning in formal, machine-checkable forms.
Mathematical capacity, which is the trained ability to verify, interpret, and challenge mathematical reasoning, is not a byproduct of theorem production but a form of infrastructure, built over generations by institutions that cannot be reconstituted on demand.
- sachaa
What worries me isn’t AI replacing experts, it’s that we may stop producing people who know enough to notice when AI is confidently wrong.
- mondrian
It sounds ‘the singularity is near’ not so much because AI is reaching escape velocity forward, but because we’re systematically pushing humans back beyond the threshold where computers are legible?
- titzer
AIs should be forced to show their work. Every tool they use, every program they generate and run in the background, and every logical inference. They should be forced to produce Lean or Rocq proofs or execution traces for all the computation they use. For facts, they should be able to produce sources. For any abstract reasoning, they should be able to break it down into explainable steps.
Then, on top of that, they should be able to explain any of that, at any level of detail, whether talking to an expert or a layperson.
- MSkill1
I'm not exactly sure how you would go about grading mathematical proficiency. I went through calculus two and discrete mathematics, but I'm sure that there are things I have forgotten now even though I would be considered familiar with most leading edge AI technology. If I'm being honest, I'm not sure I could pass the final exams I took to get my CS degree right now.
- pxagntuvzt
I strongly suspect most people already live in a world, that is not reasonable to them.
If you observe most policies to come from authority, you might not feel the loss of your own capabilities as much. Most people who are subjected to rules of law, bureaucracy, politics, or religion, might see these rules as arbitrary.
If you think of the way math gets taught in a this-is-right-everything-else-is-wrong way, it will evoke the same feeling of arbitrary authority.
Playing around and exploring math is usually only sometimes taught to children from academic/STEM families. So why would someone mourn the loss of capability for mathematical reasoning in people, if they only experienced it as following something incomprehensible anyway.