Does SpacetimeDB Scale? Breaking Down the Three Dimensions of Scale
Ok, but Does It Scale?
SpacetimeDB's CEO answers the most common question about the platform: how does it scale? The post breaks down scaling into three independent dimensions—compute, storage, and networking—and explains that while horizontal scaling works well for storage and parallelizable workloads, it often fails for transactional workloads with contention. Using Postgres, Neon, and CockroachDB as examples, the author argues that distributed databases like CockroachDB can be slower than a single node under contention, and introduces SpacetimeDB's approach: high performance under contention with tools for scaling parallelizable OLTP workloads.
The ugly truth is that rather than doing more with more computers, horizontal scalability can often mean doing less with more computers: conceptually what one computer can do in 1 millisecond, 10 computers can do in 100 milliseconds.
- timssopomo
Spacetime sounds like really interesting technology, but I'm not sure that the comparison between CRDB is a good one.
I used to work at Cockroach Labs. The problem it's solving is fundamentally different. CRDB as a solution makes sense when you need to _guarantee_ that transactions are serializable and durable, and that your application can survive node or region failures while maintaining consistency. In a naive deployment it's significantly slower than operating on a single core, but that's the price that you pay for the ability to survive node loss without data loss.
I don't see anything that indicates how spacetime solves the core problem CRDB does, which is guaranteeing that single node failures can be tolerated with zero data loss or loss of availability. It sounds like transactions by default are required to be written to disk before completion, which makes them durable on a single node, but you can't ensure they're consistent across nodes without accepting the network overhead and losing transaction throughput (on writes, anyway).
Also, FWIW, in the several years I worked covering basically every incident, I can't recall seeing a network-bound cluster. Like anything else, there are tradeoffs. You give throughput, you get consistency and availability, and you don't need to engineer how to avoid data loss or availability with node failures. Unless I'm misunderstanding, spacetime is solving a totally different problem.
- chermi
Off topic but... I had a friend who's main takeaway (tongue-in-check) from harvard mini-mba while he is in a SEAS program was that all you had to do to look smart around VC/tech-oriented business people was to ask this question.
Then we actually founded a startup and realized this was not a silly question. But perhaps that's circular when the default path is VC funding, hmmm.
- philippta
I benchmarked Postgres in single- and muti-node setup against CockroachDB a while ago and my findings match what this post is talking about.
https://github.com/philippta/postgresql-cockroachdb-benchmar...
- echohack5
As someone who has been using alot of spacetime for side projects (like https://heat.echohack.app), I am continually impressed with the speed at which it operates.
I think there's alot of interesting things happening in the database space. Vitess/Neki, vector stores, spacetime are all really good things to be happening. I think it's a shame that database developers seem to have a drama filled timeline out there. It's... all very exciting, together.
Anyway, Some things that need improvement (some of which are addressed by this blog post):
1. Backups (fast recovery) and Disaster Recovery (slow, durable recovery)
This is a big one, but sometimes speed is not the only objective you have to meet. You need to have certainty that, if everything goes down that you (eventually) can bring things back online. I don't really have a way of doing that today.
2. Read replicas sure would be nice for analytic workloads
3. Durable writes to s3 would be nice for intermittent bursting workloads (like ci systems)
I think the Spacetime folks have their work cut out for them, not necessarily because of the technology (that's hard too) but because the AI models have seemingly decided that Neon and Postgres are all that exists.
I think spacetime has a bright future ahead of it, and I am wishing the team all the best as they work hard to imprint something new on the universe.
- themgt
> You get to deploy your server logic directly into the database
> You may make use of the Licensed Work provided your application or service uses the Licensed Work with no more than one SpacetimeDB instance in production and provided that you do not use the Licensed Work for a Database Service.
Therefore, as an OSS product, SpacetimeDB does not scale.