AI Is Strip-Mining Mathematics, and Mathematicians Are Weeping

I can kind of relate but I feel like I am mourning something different: the loss of the desire to seek truth in tech, math etc.

Nothing is validated or tested anymore. It’s all about the shape or believability or even how loud it is.

They dropped 100+ papers no one read. I am sure mathematicians are reading them now but even this forum has concluded that these problems are solved, without reading or validating.

It’s like everyone forgot “don’t believe everything you read on the Internet.” And it makes for a very boring discussion when everyone is just spewing unvalidated claims, layering them with strong beliefs founded on similarly unvalidated claims. It’s dull, without substance and just pure noise.

> “Google Translate is amazing, but it has taught us essentially nothing about linguistics,” Strogatz said. “We don’t understand why it works. It just works. That’s the Twilight Zone. It is knowledge without understanding.”

This is not true, and it is the crux of an enormous debate we're now having across many, many fields.

Classical Chomskyan syntax (which models sentences as parse trees) has been enormously, astonishingly predictive of the rich structure in real-world human language. But we are quick to forget that we invented concepts like "words" and "noun phrases" specifically to help us analyze and predict things about human language. What LLMs and modern MT shows, definitively, is that there is another, totally separate way to describe these phenomena, which is both vastly less descriptive, and also vastly more predictive.

That these appear to trade-off is unfortunate, but it is NOT the case that we "don't understand why it works." We absolutely understand why it works, if we didn't understand we would not be able to improve these systems. What we do not have is a clean, descriptive theory that boils this down to something that is formally specifiable and analyzable. But those are different things, and this is the root of the disconnect between many communities working in and around these spaces (linguists vs computational linguists; mathematicians vs computational mathematicians; mech interp people vs empiricists).

This piece circles, without naming it, around the concept of “deontic scorekeeping” central to the work of Robert Brandom on semantics. The idea that our ability to create and sustain conceptual systems depends on the give and take of reasons within a social setting. Brandom has been front of mind for me lately because elements of his theory both explain why LLMs should work in principle, and where they are bound to be radically deficient. An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time.

Worth reading: Robert Brandom, “Making It Explicit: Reasoning, Representing, and Discursive Commitment”

I think Thompson has no understanding of endeavors like pure math or the theoretical sciences. In the words of Einstein, "I want to know how God created this world. I am not interested in this or that phenomenon, in the spectrum of this or that element. I want to know his thoughts; the rest [is] just details."

I think mathematicians aren't bemoaning their lost sunk costs. They're now aware that they may never again have the joy of being the first to pull up the edge of the tarp of ignorance and see the raw truth beneath _for the first time_. Their voyages of discovery, which give life meaning, may never leave dry dock again.

I still think mathematicians are going to be as important as ever. The LLMs are only finding what was technically possible given our current breadth of knowledge after brute force solving problems. Eventually it will run out of things that it can do with its training and the only way to get it to do more will be training it on new human found knowledge.

Finding that new knowledge may be exponentially more difficult, but it will be much more valuable.

Knowledge without understanding would be just getting "42" and not being able to ask any follow-ups.

But I don't think that's where we are at the moment. I believe the results, proofs and papers can be reviewed and understood, just like if a human peer had published another result.

I also think that the speed at which results can be now generated can't be followed by our collective speed to understand them, using the methods we are used to.

As we enter the next phase where AI publishes using mostly references of other AI results, this will compound exponentially until it would take any human decades to verify a single result by hand end to end.

I agree translating a phrase doesn't teach you a language, and I also believe translators are an essential tool that results extremely helpful when learning a new language.

So instead of throwing the towel, could we develop and adapt ways to improve and aid our understanding and verification processes?

As a software engineer it is hard for me to feel sorry for these mathematicians. Software engineers are getting laid off because of advancements made by AI. Math researchers are employed at universities, and universities aren't laying off mathematicians now that OpenAI has solved some of their puzzles. Mathematicians can still feed their families, software engineers displaced by AI can't. Yet I see far more of these blogs about the plight of mathematicians, not enough about the plight of software engineers.

I think people vastly underestimate the ability of human to respect human performance despite being inferior to machines. Think about chess players vs stockfish, marathon runners vs a car, etc.

I think the crux is much more prosaic, as buried in the article.

> I also don’t want to lose my job to a tangle of GPUs.

To put a GPTism, it’s not about the knowledge or status, it’s more about keeping their job and money.

And that can only be answered by policies, and imo by UBI. So votes if you can people, protest otherwise, discuss, debate, lobby with your friends, families acquitances.

I think an under appreciated part of mathematics has been contributing clean and elegant work precisely so that others could use that as a foundation for further work and new insights. This dump from Open AI seems to be the opposite. Huge and messy, incomprehensible and taken seriously only because of Lean.

There is this assumption that AI will contribute to faster progress in math but given the garbage dump of proofs which we have to take on faith are robust I wonder if the opposite is true.

My reservation could only be dismissed if AI turns out to be able to contribute fundamental new insights and pose new problems. This takes us into an area where AI progress has been slow to say the least.