The Four Horsemen Behind Thousands of Postgres Outages
The four horsemen behind Postgres outages
I explore the four critical issues causing massive Postgres outages: VACUUM inefficiencies, rigid connection limits, unpredictable query plans, and poor JSON handling. Drawing from real-world startup experiences, I detail how these architectural flaws lead to downtime and introduce pgrust, my Rust-based reimplementation designed to solve these problems with 64-bit transaction IDs, a thread-based model, and an adaptive planner.
The difference between a good query plan and a bad one can be the difference between a query taking 10ms and taking 10 minutes.
- bananamogul
This article states multiple times that spinning up a process is expensive/very expensive. Is that really true? I ask out of ignorance.
“Compared to other ways of doing parallelism, processes are very expensive, both in terms of taking CPU resources but also the amount of time it takes to spin up a new process.”
“Because it’s expensive to spin up new processes, Postgres will only do this for long-running queries.”
Also:
“There’s been years of people talking about switching Postgres from a process model to a threading model, but nothing concrete has come out of that.”
I’ve read several times that on Linux, the cost between a process and a thread is relatively small.
- dmitrijbelikov
If you understand Postgres's problems so well (though in reality, such problems can occur in many applications, and it's unclear how your rewritten version attempts to solve them), then why not address them in the main branch by becoming a contributor.
Less code and more value.
- spongebobstoes
the ability to even attempt this ambitious project is what free software is all about