Gwern Branwen steps back from full-time writing to launch Guardian Angel Inc
Gwern reties from fulltime writing to launch Guardian Angel Inc
Gwern Branwen, known for his extensive research and writing on AI, has announced he is stepping back from full-time writing to launch a new venture, Guardian Angel Inc. The company aims to develop AI safety tools, reflecting Branwen's long-standing interest in the field. The announcement was made via a tweet, which is currently inaccessible.
We're unable to show this content. The content may be private, deleted or only available on the app.
- wcfrobert
A few snippets from his full post here: https://gwern.net/guardian-angel.
> "The chatbot personas are deeply misaligned with you, and aligned with their owners; and the economic incentives are to farm you with ads and subscriptions, while racing not to amplify you but to replace you."
> "On my visits to the Bay Area, I would ask AI researchers or interns why they are doing their current research or projects, when in a year or three agentic LLMs could probably do them; they rarely had a good answer, or any idea what they would be doing in 3 years"
> "One programmer driving 10 Claude instances, because he has to review their work, will never be as valuable as fully autonomous Claudes where there can be almost arbitrarily many instances, like 10,000 instances… but such scaling requires removing him from the loop as much as possible. And this is true of everyone else, whether lawyers or writers or researchers: increasingly, you are the bottleneck to be optimized away."
I fully support the 3 core principles of GA: (1) Enhancement, not replacement (2) Mental Sovereignty (3) Self Actualization, which I think is a path to a more humane future.
- keiferski
And this is true of everyone else, whether lawyers or writers or researchers: increasingly, you are the bottleneck to be optimized away. As long as human workers cannot be removed from the loop, the AI tools are complements, but as soon as they can be, there’s no reason to keep them, and trillions of reasons to substitute AI for them. (And once human workers are no longer irreplaceable, where does their power or relevance come from?)
There are so many false assumptions and leaps of logic here, I don’t know where to start.
The idea of a more user security-conscious LLM is a good idea. I don’t know why it needs to be wrapped in novel-lengths of LOTR quotes and Sci-fi speculation.
That said, it’s very interesting to juxtapose this post with others by writers that are interested not in “maximizing product output” but in actually expressing themselves.
This post is like the ultimate expression of the idea that the only valuable thing writing has to offer is information, and not personal experience, poetry, advice, humor, etc. - which I suppose is fitting for someone that has written anonymously for over a decade.
It’s a shame that he didn’t reach the opposite conclusion, like some other writers have – the way to respond to LLMs is to be more human, not less.
- sillysaurusx
(I'm not a part of GA.)
I've known gwern for the better part of a decade. Working with him has been great. We've done quite a few projects together, including being the first ones to demonstrate that GPT-2 could play chess (or rather, can be used for actual useful work instead of just being an autocomplete).
He's a great person. I've wanted to do a writeup on it for some time, but what surprised me the most is his humanity. He genuinely cares about the implications of his work. But beyond work, he also cares about the people around him, and it shows.
Just wanted to put in a good word in case someone here was on the fence about applying.
As for GA itself, I think it's an ambitious idea worth pursuing. Imagine an LLM which actually sounded like you, and to an extent, thought like you. How much would you pay to have access to a smarter version of yourself? So the idea is solid, and early results seem promising from the samples I've looked at.
They're also taking personal info very seriously. Obviously, I can't make any promises of what they will or won't do. But they've spent some time studying questions like "What if someone adversarial has access to my GA? Could they get my bank account info?" and came up with a technical solution that I really like.
- rocmcd
It's hard to read Gwern's accompanying article without seeing this for what it is, which is a kind of mania.
I'm sure he means well and is genuine in his aspirations, but what's outlined for GA is framing LLM's as quasi-gods, which they absolutely are not. I wish him the best, and look forward to being proven wrong.
- kashyapc
> What would it take for LLMs to make me 100× more productive? Without this, I am doomed to irrelevance.
Are you, though? You will only be "doomed" if your place your value system squarely on "productivity". Then what is to differentiate you from a machine?
* * *
Edit: I'm not sure how I can feel confident about a proposal that puts so much value on "productivity". How can you reconcile this with "self-actualization"? (Don't get me wrong, I like my LLM-based productivity gains as the next person, but I care more about wisdom than becoming "100x more productive".)
Edit 2: "the goal of GA is to preserve individual human cognitive liberty and flourishing" — so the proposal is to do that by overlaying a software bot that continuously mimics your "self"?
- jvanderbot
Their profile specifically says they will not acknowledge follow requests, but they only share posts with followers. As such, not sure what the value of the top level link is.
- malshe
> The big AI labs are building a single mind for everyone
Reminds me of Pluribus
(I just started watching it on Apple TV so maybe this is a late realization for me)
- jephs
Only, if he had instead fallen in love with the version of this idea in which a community acts as the principal, rather than an individual.
(I would like to be known to the agent serving my family, that serving my friends, my team at work, the PTA at my kids' school.)