When Genius Fails: The Intellectual Arrogance of the AI Labs
Leopold Aschenbrenner's $20 billion hedge fund, Situational Awareness, blew up, and the author—an ex-hedge fund guy with an AI background—sees a pattern: the frontier AI lab culture's lack of intellectual humility extends beyond money management. Drawing parallels to Long-Term Capital Management and Malthus, he argues that AI experts overstep their expertise in economics and other fields, leading to hubristic predictions and risky bets. He also cites examples of AI labs' overconfidence in areas like materials science and radiology, and the irony of Hugging Face using a Chinese model to defend against an AI attack because American guardrails were too restrictive.
Sure, going all-in leveraged long Nvidia in early 2023 has been great and all, but the burdens of history are heavy. I would not choose this.
- mjr00
> It’s much easier to say someone else’s job is going to be fully replaceable by AI when you don’t actually know what they do.
Too true. This isn't limited to AI, either. The most obvious example in my lifetime was during peak blockchain hype, when people who had never worked in finance convinced themselves that blockchain was going to act as the backbone for how money gets moved around. As if the problem that needing solving was Bank of America doesn't trust Capital One to update a number in their database.
The nice thing about AI, at least, is I can always push back and tell people, "Sure, we can do this with AI. I just need you to use Claude or ChatGPT manually to prototype how it would work." This normally results in the requestor realizing that there's human judgment calls involved in the inputs, process, or outputs that require meatbag intelligence.
- sarreph
What the author is getting at here:
> Climbing Mount Everest and getting a PhD are both hard. I wouldn’t say that doing one means you can do the other “no problem.”
and here:
> Surgeons and electrical engineers may be extremely smart and accomplished, but that doesn’t mean they have the ability to weigh in on climate science.
Feels tangentially like a version of Nobel disease[0], discussed here[1] a couple of weeks ago.
---
- skrebbel
Wait is this real? Did a random 25yo get 45B under management because he got some interviews after getting fired from OpenAI? I missed all of this and it almost seems like performance art to me.
- augment_me
Does the actual outcome matter or does the promise/what is sold/pushed upon society matter?
Was social media a clean win for human social interaction? Was fast food the replacement of food that was promised in the 70s?
Were doctors and shrinks in phone apps rather than real life a clean win for diagnosis, understanding and improvement of life?
But society was completely shaped by the market in these questions anyways in a way that is impossible to deny.
While the author might be right about the arrogant PhDs imposing their view on other fields they know jack shit about, it doesn't make it less true when you have billions backing it and making it a reality, it will eventually be it, even if it's arguably worse.
- cmiles8
There’s a tremendous amount of naivety on display by the big labs.
A bit of that is to be expected from any disrupter, but there’s now serious questions about if these startups have the basic competencies required to survive long term.
- amelius
Working in AI was once reserved for intellectually gifted people. But with the recent advances in compute performance, working in AI today is basically not much different from pounding on a black box as long and as hard as necessary to make its output correspond to what you want it to be.
- miguelspizza
The worst materialization of this are the slew of startups that are going to “watch your employees work so they can learn their job” as if random clicks on a computer can be reversed engineered to automate someone’s job.
This sort of audacity is a large part of the reason people increasingly hate AI
- mrhottakes
This accurately describes a large portion of the people working in "tech", and it doesn't look to be getting better anytime soon.