A Million Tokens a Second Is Not a Million Times Faster Work
What if AI worked at 1.000.000 tokens per seconds?
A million tokens per second can mean context capacity, input processing, aggregate throughput, or single-agent output. Even at that imagined speed, a one-minute check can dominate total time: 800,000 tokens written in 0.8 seconds still take 60.8 seconds end to end. The real bottleneck is judgment—taste, specs, tests, physical evidence, and fidelity—not generation speed.
Whatever you do not speed up becomes nearly all the remaining time, the logic of Amdahl’s law.
- gchamonlive
I don't want to bash on the article, but a much more interesting take with llm that worked at 1kk TPS would the the countless amount of real time applications you could build with it.
I like electronic music, I dance to it a lot. What if I had a LLM that had enough throughput and low latency to do the reverse, take my dancing and generate music.
We are still stuck stargazing raw artificial cognitive intelligence, but real progress will be when we stop perceiving these as external intelligence and just an extension of ourselves.
- sixtyj
“If” is temporary. More TPS is the question of time. In such usecase human in the loop will be eliminated as nobody, even anybody with fastest recognition, will be the slowest part.
How many industries are? How many use cases are possible?
If you solve everything, what will people do? Sitting on the beach, sipping drinks?
Massive degradation of human brains and rise of neurodegenerative illnesses as unused device starts to be erroneous?
1,000,000 TPS is good for some use cases but not publicly available…
- ed
At 1m tps, LLM’s could generate UI’s in realtime (a 5k webpage would take 5ms). Applications will have every little in common with today’s stacks when that happens. (For starters, rendering, event handling and data storage could move to the context.)