AI Is Throwing a Roadside Picnic
In the Strugatsky brothers' novel Roadside Picnic, aliens leave behind incomprehensible artifacts that spawn chaos. The author argues AI is now doing the same to science: labs are solving Millennium Prize problems and dumping discoveries faster than researchers can understand them. This gap between answers and understanding could become the biggest risk of AI-driven science.
The shape of things to come: proof will become cheap and understanding will become the bottleneck.
- asimpletune
> All this new knowledge is arriving at a pace much greater than the field’s ability to process it
That’s the whole issue: it’s not knowledge, it’s data. AI can not provide an account for the proofs, or be questioned about them, or organize a talk exposing the idea because there’s no understanding behind any of it.
This is what Tao has been saying; that the results, in the strictest sense, were never the goal. It is ultimately the understanding (that previously was required to form a proof) that is valuable.
- prodmod
Usefulness relies on understanding, even if that means understanding what is possible to ask for. What is being generated right now is really just a translation from one inscrutable phenomenon into another. In neither case do we have power over the data, because we don’t know what it means, truly where it comes from, and even the beginning of the limits of what the discovery allows. We’ve traded one black box for another, and still need to unpack it.
- pavel_lishin
I think Ted Chiang's story "The Evolution of Human Science" may be a significantly better metaphor here. What AI is dropping is (allegedly) just answers, not mysteries. In both cases, we don't really understand it, but in "Picnic", most of the things left behind aren't even useful, merely mysterious or more often dangerous. In "Evolution", answers (and only answers) and technologies arrive pre-packaged, with zero understanding.
- https://en.wikipedia.org/wiki/The_Evolution_of_Human_Science
- https://gwern.net/doc/fiction/science-fiction/2000-chiang.pd...
- skybrian
I like this metaphor better: the mathematics community is like a group of ancient astronomers that are unexpectedly getting the output from a modern telescope. "Why even look at the sky anymore," some say. "And we don't even own the telescope!"
Meanwhile, the Rubin observatory is creating an archive that grows by 20 terabytes each night, and nobody really worries about this. Dealing with that amount of data is definitely a problem, but also a nice problem to have, provided that there's a scientific community that can deal with it. Mathematics has unexpectedly become Big Science and there needs to be a better relationship between the funders of the scientific instruments and the scientists themselves.
It's a bit confusing that the output from the scientific instrument looks like badly-written academic papers. Maybe it should be treated more like raw data? The format is not ideal, but historians (for example) often have to deal with far worse.
- m3047
I like that metaphor. But what it calls to my mind is an "alien artifact" which was found by a road, and "how did these refined metals get placed inside of a rock, in such an unnatural formation?" and there was much debate about this artifact. Years later it was determined that it was in fact a spark plug which had been remineralized by the effluent from a mine upstream: an unusual process acting on a discarded object, and people who want to believe. Nobody thought that the remineralization process could take less than hundreds of years, which ruled out "spark plug".