AI's Real Advantage Over Mathematicians Isn't Intelligence—It's Memory

It's also "out-brute forcing them." It just never gets tired. If a mathematician picks a research direction and spends a whole week on it and it doesn't pan out, they will likely be annoyed, need a break for a while, etc. This thing just does not ever get tired or discouraged or care; it's just onto the next thing until something ends up working.

One thing about human mathematicians is that they only publish positive results. Professors etc might have file drawers full of "negative results", but the incentives and bandwidth of human mathematicians makes publishing these useful results impossible.

But AI agents have no such limitations and can publish and re-use negative traces easily. There have been some recent projects (https://www.theoremdb.org) aimed at exploiting this fact. https://news.ycombinator.com/item?id=49227505

In general though, LLMs do not have the same limitations and incentives as human mathematicians, and the next year's tsunami of change will make this abundantly. clear.

I can't help but think of Michael Nielsen's essay "Augmenting Long-Term Memory" [1].

> Many people's model of accomplished mathematicians is that they are astoundingly bright, with very high IQs, and the ability to deal with very complex ideas in their mind. A common perception is that their smartness gives them the ability to deal with very complex ideas. Basically, they have a higher horsepower engine.

> It's true that top mathematicians are usually very bright. But here's a different explanation of what's going on. It's that, per Simon, many top mathematicians have, through hard work, internalized many more complex mathematical chunks than ordinary humans. And what this means is that mathematical situations which seem very complex to the rest of us seem very simple to them. So it's not that they have a higher horsepower mind, in the sense of being able to deal with more complexity. Rather, their prior learning has given them better chunking abilities, and so situations most people would see as complex they see as simple, and they find it much easier to reason about.

I once tried out his Anki approach during a math lecture. Whenever I reiterated a card, say about some lemma, I noticed something interesting about it. This was delightful and many lemmas became much more streamlined over time. It's not a "solution" to mathematics, but I found it delightful while it lasted (before akrasia or lack of time kicked in and I stopped doing it).

[1] https://augmentingcognition.com/ltm […]

It's probably worth checking the site history before posting. The guy who wrote this looks like some kind of "race science" crackpot.

I thought this was fairly obvious. I don't consider any of the AI models I've worked with particularly smart, but they've read orders of magnitude more than I ever could in my entire lifetime. They have far better book knowledge than I have, so that's how I use them. I use them for things that I suspect other people out there would know, but I don't. But when I work on something that I suspect is truly new, the models rarely understand what I'm doing, and I've got to do it myself. Although I still poll them for basic principles, best practices, and other advice.

LLMs are still missing a part of working memory. Part of working memory is being able to attend to small amounts of information and then understand and parse all the pieces of that information. When LLMs use their "working memory" they just analyze different probabilities of tokens and there is no prioritization or understanding of the information in the way humans have it.

If there is no training data or data in the context that leads it to the correct result then it can't do it, whereas a human seems to be able to generalize and abstract a goal and then repeat an action or thought process in a 'recursive' manner to reach the result. AFAIK LLMs don't do this.

Just as an example to illustrate. I recently asked an LLM to organize a bunch of artists albums into whether they were released by a major label or an independent label, and for the most part it did a good job. But there were albums that it classified as independendent that weren't. I presume because it either didn't run into the right data when searching or it misunderstood the data it did find. A human would not do this because if a human had a list of all major labels, it could instantly detect whether an album was or wasn't indie, because it doesn't do any complicated parsing or token probabilities that LLMs do, it just recognizes a pattern (either an album is indie or it is not, a human brain needs simply one piece of information to decide this), an LLM is not that simple.

In a way human brains are simpler than LLM […]

While TFA itself makes sense I disagree with the title and the conclusion. I would not consider referencing working memory during thinking as “remembering” but as a part of thinking itself. Working memory is the RAM to the much larger but higher latency indexed database that is our long-term memory. As such I would say AI is out-thinking us, even if in a brute force sort of way.

I think where you could say it is out-remembering us is when it can contemplate the vast universe of patterns, gleaned from essentially all human disciplines, encoded in its weights, that may let it draw connections that a human could not, unless they just happen to be familiar with multiple disciplines.

Which is why I think the analogy with Von Neumann / Einstein is also a bit off. From TFA it seems Von Neumann was more akin to what AI does, than Einstein. I don’t get the impression that it was Einstein’s memory but his ability to look at things from a radically different perspective. So far I don’t know that we can categorically say that LLMs can or cannot do that.

Sounds like another attempt to frame AI in a way that makes them feel better about themselves.

The simpler explanation is that a working memory is a requirement for intelligence, and a larger working memory will make you more intelligent. Hence the AI can in fact be more intelligent than the mathematician.

There are plenty of high value endeavors where being a superhuman knowledge remixer is right on target. But even capturing all of the knowledge is proving elusive.

I use coding agents. I think they're pretty good overall. They save me a lot of tedious coding. For example I probably wouldn't spend the time to implement native splash screens for all the build targets of a Flutter app, but I'll have the coding agent do it.

Nevertheless, for all the time that we've had coding agents, it's still trivially easy to find the jagged edges of their training. For example, Gemini evidently doesn't know if the Xcode part of a Flutter tool chain is misconfigured. That's not exactly a Millennium Prize problem. But it is shaped wrong for a training set for a coding agent.