OpenAI Released 700 AI-Written Math Papers, and a Mathematician Saw a Package from Mars
Who Taught the Machines Mathematics?
After OpenAI released over 700 mathematical manuscripts produced by an internal AI system, a former FOCS program chair recalls the excitement of receiving 400 human submissions—and the disconnect he felt this time. He questions who will verify the results, how errors will be attributed, and whether the unwritten knowledge mathematicians pass down is being extracted to build proprietary systems that could overwhelm the profession's capacity to check them.
I would call this a form of extraction, though not a simple one.
- stabbles
I'm reading this paragraph with the precedent of AlphaGo in mind, where none of the Go players would have expected that databases of human games would be valuable training material for machines instead of humans:
> I learned mathematics because others gave their time and knowledge freely, and I wanted to do the same for the next generation. The possibility that a student would go on to do something I could not do was part of the point. But I had imagined this as a continuation of a human activity. [...] I could not imagine treating this knowledge as something to be guarded.
But soon after, AlphaZero was trained purposefully without human games in the training data. The "extractive" training was sufficient to show that a certain machine learning model worked, but unnecessary to produce a machine that performs beyond the level of humans.
For coding agents I imagine it is similar: humans introduce so many subtle bugs, and they're only subtle relative to human cognitive capacity. I'm sure subtle mistakes pass peer review in mathematics too, even if it's generally more rigorous.
What I'm getting at is that labs have demonstrated that a certain model architecture is capable of producing mathematics at a (super)human level, and their extractive methodology was sufficient, but possibly not necessary. You can fight in court whether their methodology is against the law, but it would be naive not to look beyond that, which is my main gripe with the AHM statement the author refers to [1] […]
- glimshe
One thing I don't yet understand and would appreciate an opinion from a professional mathematician is: are we okay with mathematicians that outsource the discovery of new theorems to AI while providing verification services themselves?
For instance, imagine a mathematical institute that hires reasonably competent mathematicians who are amazing AI prompters but lack mathematical creativity or "genius". All they produce are mathematical papers with clear proofs and explanations. Very human readable stuff. But the catch is that they are not discovering any of it. All they are doing is "translating" full AI discoveries into a format others can understand.
Is that acceptable?
- hypfer
Is it simply a bias in the sampling or are the people in math (the discipline) as shaken as the stuff that shows up on HN makes it look like?
FWIW, it reads very similar to the woes of the SWEs previously, in the sense that it kinda appears like people assigned value to the wrong part of the work within the discipline?
The take-away of it being it being imperative to have open weights models accessible to all, instead of cloud-hosted models owned by few is of course true regardless of those specifics, but it gets a bit muddied by that, which is a shame, as that will be weaponized to dismiss it.