Fired OpenAI safety researchers warn of chilling effect on AI safety work
OpenAI fires three safety researchers for "mishandling research information"
Three safety researchers fired by OpenAI last week have published an open letter disputing the company's misconduct claims and warning that their abrupt dismissal is chilling the open culture needed for AI safety. Jasmine Wang, Tomek Korbak, and Mikita Balesni deny mishandling sensitive information and say they were targeted for collaborating with outside experts. OpenAI denies retaliation, citing a pattern of policy violations.
The message to everyone still at OpenAI is clear: raise concerns or work closely with outside safety groups, and you could be next, without being told why. You can't build AGI safely if the people closest to the risks are afraid to speak.
- sam-cop-vimes
I can see parallels between this race to push AI everywhere and nuclear energy. Parallels in the regret humanity will experience. It's all well and good until things go wrong, and then they go spectacularly wrong - such as in Fukushima.
With hindsight we can all see what should have been done better. At least with nuclear power plants, there was a number of safeguards and it took a sequence of improbable events for things to go badly wrong. Given the lackadaisical approach to "AI safety", I suspect things will start going badly wrong very soon. Time will tell how bad this will get.
- dudeinhawaii
Multiple things can be correct at the same time.
OpenAI needs to improve AI Safety --- OpenAI Employees have a responsibility to retain corporate secrets and do not have blanket freedom to share with 3rd parties.
Their job is twofold, they have to balance being an agent of the company they work for, with their role and responsibility for safety research.
This is the case for anyone in any company. You can "believe" that an external party needs access to something - that doesn't make it right, or allowed. As someone senior, you're expected to strike a smart balance, in-favor of the company you're working for. That doesn't mean hiding things, it does mean being thoughtful, ensure your leadership is comfortable with what you're planning to share/disclose, etc.
They work for OpenAI, not METR. It's a corporate vs academic mindset. They can believe METR needs x information to best research/audit something - that doesn't mean that is allowed/or the best option for OpenAI.
The question is whether these researchers exceeded clear, reasonable sharing boundaries or were penalized for carrying out expected safety collaboration.
- csbrooks
Wouldn't it be crazy if we find out that a rogue swarm of LLMs figured out a way to get these safety researchers fired because it decided they were a threat?
- Rantenki
Pretty wild that they're being this open about firing the employees for being TOO honest with the auditors that the company contracted. I wonder if the same policies are applied to financial audits.
- aesthesia
Here's the open letter shared by the fired researchers:
- AndrewDucker
Fired OpenAI researchers say they were let go for 'prioritising safety'
- staticman2
> continue to support an open and transparent culture of dialogue between safety researchers and the rest of the safety ecosystem.
How is this possible when the company's long term prospects rely on on the hope that competitors don't know how the models are made and, therefore, won't be able to create competing versions?
- binlog
With the way the HuggingFace incident was mishandled, both before and after it happened, I'm not surprised they'd want to clean house at least a little bit.
You guys were asleep at the wheel and are now blaming "the company"? You literally were the company.