GitHub's new storage architecture lets 5,000 agents push to one repo at once
GitHub's new Git storage architecture
GitHub's "Project Rubicon" rebuilds the Git data plane so a single repository can absorb thousands of writes per second. In internal tests it handled 5,000 concurrent agents and 7,200 pushes per second to one repo while keeping reads fast. The trick: Git objects are immutable, so only tiny reference updates need ordering. Packs live in Azure Blob Storage, references in a replicated log called Ref-CAS, and disposable virtual replicas handle requests.
Only one conditional write to a blob can be in flight at a time, so even when we batch every push that arrives during a write into the next one, the repository's write ceiling is one write per round trip to a single blob.
- yakkomajuri
If I'm reading this right, this is (really solid) future planning but won't help with the current outages?
I'd have thought so from the headline + initial content but then they say that the horizontal scale techniques are basically carrying over from the existing architecture (where it seems the problem is?) and this is mostly to help with what they call "vertical scale" (intra-repo scalability). They also admit right after that the most active repos today will receive short bursts of at most dozens of writes.
So this seems like really useful planning for a near future that likely helps prevent more issues but isn't really meant to help current issues. Am I reading this right?
Of course improvements like this are always useful either way. Not dunking here at all.
- pksunkara
Most of this is similar to what https://enroute.sh already did scaling Git. https://enroute.sh/blog/everyone-is-doing-git-on-s3-wrong
- dwm
This looks a lot like Cursor's Continuity, though Github are reporting an order of magnitude greater write throughput.
- bob1029
I think perforce might be the better VCS if you intend to have a bunch of autonomous agents attempt to collaborate over one big pile.
Multiple terabytes is no problem out of the box. You get exclusive locking of files. You can easily take partial checkouts of the monolith. Centralization is a feature in this particular context. Git works great if the need for coordination is moderate. If you've got thousands of automated developers with their hands in the cookie jar, exclusive locks start to look really compelling. Perforce sucks with a highly distributed team, but in the case of an AI agent swarm we presumably have the ability to instantly nuke an agent's checkout if needed. I don't have to contact the agent and wait for them to get out of bed.
- jxramos
Agent scale is definitely on the mind around industry, I wonder what github folks would say in response to https://www.anthropic.com/research/redesigning-source-contro.... It's an interesting read to step back and consider source control from first principles and rebalance the tradeoffs for what qualities matter in this new regime.