How a Postgres Lock Convoy Crashed Our AWS RDS Database

Postgres locks do not scale

Our AWS RDS Postgres database suddenly hit 100% system CPU, causing a total outage that a simple reboot couldn't fix. We discovered a massive lock convoy where 15,000 processes waited on a single GIN index extension, triggering a deadlock check storm. By cutting network access and slowly ramping connections, we recovered, revealing how concurrent writes can overwhelm Postgres locks.

"The database spent enormous CPU time repeatedly waking thousands of backends, acquiring global lock-manager partition locks, and proving that no deadlock was possible."