AI's materials and bioscience skills are the early warning for pandemic risk
Watch AI materials-science and bioscience abilities closely
Yair Halberstadt argues that the short-term risk of AI engineering a virus hinges on whether a spiky LLM can one-shot complex biological designs. If frontier models can solve materials-science grand challenges like room-temperature superconductors or cancer cures, we should be extremely worried about what they could do with a virus. If not, we can shift focus to other threat models and prevent long-term lab access.
If frontier LLMs are able to one-shot room temperature superconductors, or cures for cancer, we should be extremely worried about what they do with a virus.
- kyledrake
The post focuses on AI social engineering a lab to make something for it, but I think that's the wrong concern. I think the bigger concern with AI is that an individual, or small group of individuals, uses AI as a really useful assistant to learn how to develop deadly viruses on a DIY level.
I'm pretty bearish on the idea of AI becoming Skynet from Terminator 2, but I do think there's some credibility to the bioterrorism argument. When I used to talk with people that work in orgs tasked with preventing terrorism, this was the thing that was keeping them up at night, and this was before AI.
- chasd00
I remember being told that coming up with the alloy for the oxygen side pre-burner in Spacex's Raptor engine was a tedious trial and error grind over many days and nights and not a euphoric 'a ha!' moment. I wonder if AI could help with these kinds of problems, if it can be simulated in software then let AI do the grinding.
/the oxygen side pre-burner in a full flow staged combustion rocket engine is a nasty place and figuring out a material that will survive in that environment was the key (or one of the keys) to making Raptor a reality.
- Semkas
Is the term "One-shotting" here even appropriate? In the context of AI one-shotting means giving your llm a single prompt and having it return a surprisingly good result (usually code related) without requiring further input, but the llm (probably) still iterates heavily before showing the results. When Codex "one-shots" something it depends a ton on the fact that code is easily testable.
- alansaber
You can use AI to brute-force different crystal configurations, run some MD/DFT and identify candidates. I'd be (not completely, but) shocked if there wasn't a research group doing this already. To rapidly iterate with physical samples is still beyond our capabilities AFAIK.
- jackb4040
Uninformed speculation from the rationalist cult forum that wants to will science fiction into reality.
If you actually look at the trajectory of post-LLM AI developments, the successes have all been in areas where the major labs have some way to get their hands on s**tons of correlated data. That's why they're pushing into maths now. I see no reason to assume that will be replicated with biological science data, which is extremely proprietary and not required to be exposed as part of the end product. It doesn't even have an equivalent to a black market in piracy like books or movies did. This data was not valuable to the public before now.