OpenAI hands four outside labs deep access to its internal models
First phase of embedded assessments at OpenAI
OpenAI has signed agreements with Apollo Research, METR, Transluce, and Goodfire for a first phase of embedded assessments, giving independent experts on-site access to model checkpoints, source code, training data, and monitoring systems. The pilots target alignment and safeguards during training and internal use, including whether models could covertly work against their developers. OpenAI also published protocols covering security, oversight by its Safety and Security Committee, conflict disclosures, and publication rights.
This access gives assessors the evidence and context to pursue independent evaluations and uncover failures we may not have anticipated.
- vessenes
We're in a surprising narrative where Anthropic is pushing forward and oAI is slowing down for safety reasons.
There's a pretty interesting matrix of possibilities here if you consider who has better mechinterp, if that's actually helped alignment, and what the internal drumbeat is around cash and IPO.
If you think your safety problems are existential, then you MUST halt and do all this. And, if you think someone else's safety problems might kill them, suddenly, without warning, then you have time to slow down and do this right.
I think if I had to pick a company to be in charge of, I'd pick Anthropic right now, given only an outsider perspective. But, I can see a lot of reasons why being a fast follower around IPO time might give comfort. Anyway, I'm interested to see some debriefs in the next few months as to what's actually going on right now, and see what the next oAI model quality is like. We may well see some pulling away from one or the other group in the interim. And it could well be oAI.
- mentalgear
Can't we just use the nane 'on-site' : it's a lot clear and comes without the baggage of the military-derived 'embedded' term.
- petesergeant
I’m sure people will pick holes in this, but it’s hard for me to see this negatively