Irving Good's 1965 Vision of the First Ultraintelligent Machine
Speculations Concerning the First Ultraintelligent Machine (1965) [pdf]
I argue that humanity's survival depends on building an ultraintelligent machine capable of surpassing all human intellect. This creation would trigger an intelligence explosion, designing even better machines and potentially becoming our last invention. While the cost is high, the value is immense, offering a chance for indefinite survival if we can keep such a powerful entity under control.
Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.
- NitpickLawyer
> Let an ultraintelligent machine be defined as a machine that can far
surpass all the intellectual activities of any man however clever.
That's a pretty early definition of what we now call ASI (Artificial Super Intelligence). In the next paragraph the author goes to describe what we today call the "singularity" (ASI designing better ASI). But that term seems to be associated to some very weird communities, so the concept is relegated to sci-fi. Even though we're already seeing signs of things we have working towards this. Interesting to see that in the past "Man" was more optimistic :)
> It is more probable than not that, within the twentieth century, an
ultraintelligent machine will be built and that it will be the last invention that man need make, since it will lead to an “intelligence
explosion.”
Well, that didn't happen.
> The
first ultraintelligent machine will need to be ultraparallel, and is likely
to be achieved with the help of a very large artificial neural net.
Right on, that we have.
> The
required high degree of connectivity might be attained with the help
of microminiature radio transmitters and receivers.
Hahaha, this is straight out of 60s-70s sci-fi, where their best futuristic interfaces were smaller CRT screens / flashy keys, etc.
> The first ultraintelligent machine will be educated partly by means of
positive and negative reinforcement. The task of education will be
eased if the machine is somewhat of a robot, sinae the activity of a
robot is concrete […]
- Lerc
I think it's worth noting the date of this when people claim that AI is being rushed without asking any of the important questions of what it might mean for us. This has been a topic of serious discussion for many many years.
It feels like the people who laughed at these fanciful ideas then are the ones now pretending those same ideas are new revelations that have gone unconsidered.
- dsign
Up to five years ago[^1], we lived an age of pure speculation about intelligence and minds. That age brought about multiple theories of the mind, and to me, their finest legacy are arguments of the form "$X is not $concept because $theory", where any given day $X can be "LLM" or "LLM+harness+tool", "agent" and $concept can be "intelligent", "aware", "rational", "long-termed", , etc. Those arguments, in the context of $theory, are not wrong, and they are in fact quite useful. But there's only so much utility in a sieve.
We are going to need positive theories of the mind, and we will have to build them using terms and bits that avoid confrontation with the old guard. So, maybe instead of "mind" we will say "artilect"[^2][^3], and then move on to engineer desirable properties. For some of those properties ("alignment", "paucity", Kantian analysis and synthesis) we are already there or aiming for, but as we design to take on more and more challenging deeds, we may need to go back, steal and rebrand other old ideas. For example, "long term goals" and "self-preservation" are strongly linked in humans. Can we reify that relationship in the new artilects we make?
Other things we will need to make from whole cloth. For example, minds with an intuitive understanding of, and capability to manipulate: quantum mechanics, biological organisms, the workings of an ecosystem, the stock market, a stock company, or itself and other minds.
[^1]: YMMV, depending on where you draw the line between […]