Hetzner Cloud's Open vSwitch network stack powers over a million servers
The Hetzner Cloud network stack – history and technical overview
Hetzner's first blog post in a series traces the evolution of its cloud network stack from Linux bridges and static routes to an Open vSwitch-based data plane. The custom stack runs on VM hosts, handling public and private networking, firewalls, DHCP, and metadata services. A homegrown controller called Flusskrebs orchestrates Open vSwitch flows, while connection tracking limits each server to 80,000 concurrent connections. The post explains why Hetzner favors host-based networking over a smart physical network and hints at a new self-built stack for better scalability and IPv6 private networks.
By running these services and network functions on the hosts, we gain a lot of freedom and flexibility when designing and building our product, and are not limited to what hardware vendors offer.
- publlus_enigma
An interesting read.
I've just started with Hetzner, and their storage box product to try cloud backups via Restic.
Whilst very competitively priced, the 3MB/s transfer rates I am currently achieving on a 1000/400 fibre connection means I am unlikely to ever use the 10TB I have supposedly been allotted.
- sandeepkd
Its interesting to learn them trying to implement the open vSwitch in python. I worked on implementing the open vSwitch in Java some time in 2014ish too. Open Stack had a lot of push back then. It was fun to gather the requirements by reverse engineering the original product based on the network traffic for something which is a middleware and has a state of its own
- marcosscriven
TIL Krebs also means crab, I always thought it was only “cancer”, so “flow cancer” seemed like a strange name.