Northflank tells customers to assume every environment variable has been decrypted
Northflank Breached
Northflank's status page reports a security incident and major outage across networking, certificates, services, addons, jobs, builds, and logs. A Wiz threat alert at 11:43 UTC triggered patching and CIRT support, and the company now advises customers to disable and rotate all keys immediately.
Assume all environment variables have been decrypted. Disable/rotate all keys.
- jonathaneunice
I hope this is a real wake-up call for teams running anything remotely SaaS-shaped. I count at least six substantial, lightning-speed attacks against multi-tenant as-a-service vendors since September. Not this year--just since last month. PingCAP/TiDB, Fireworks AI, Zeabur, OpenCode Console, Railway, now Northflank. All of them modern "we keep your content safe" players with high hopes and probably pretty good protections.
This is a full infosec emergency unfolding here. No matter how grinding it is to motivate a team to invest in protection or resilience (and it is grinding): We gotta get ready. Stat! The AI-accelerated attacker apocalypse has begun. It's boom, boom, boom, thanks for everything, on to the next. They have us sighted and are firing for effect.
So credential rotation, tripwire secrets, DFIR on standby, solid vulnerability and intrusion detection, real listening to that signal, minimized blast radius, spending caps and time limits and tight scoping on credentials, fast incident alerting... We need to get on a real war footing here.
- dangoodmanUT
A trick i've started doing is encrypting my env vars, then hard-code the decryption key in the code
that way if the provider gets popped, they need my binary to decrypt it
it shouldn't be necessary and it sounds stupid, but in cases like this, it proves invaluable (it appears they don't have customer code/binary access, but maybe that wasn't important enough to mention)
- zmmmmm
This could all go multiple ways but one of them is we end up in a place where technical security is just not possible to implement any more. What does that world look like?
Presumably it means much more behavioral security and inherent friction built in to everything we do. Maybe high value transactions will just inherently have to take multiple days to do, just to slow down things enough that human reaction time can equal what LLMs can execute?
On the technical side, I can see zero trust becoming completely saturated through every layer. The passkey you authenticate with isn't opening a gate to a trusted tier within which components trust each other - even within the same software component. Every internal function call at every layer doing hardware signature verification against continuously minted private keys (since we assume exposure, the lifetime can't be longer than milliseconds now).
It's really interesting to think through what a fundamentally robust security environent does look like if you bake in an assumption that technical security is impossible.
- dangoodmanUT
Unfortunately for infra startups, I think we're going to see a convergence back to hyperscalers as a result of incidents like this and railway.
There's no amount of "nicer to use" that can replace security, at least certainly when selling to enterprises
- hmokiguess
What is Northflank? I am surprised at the amount of new business out there
- maherbeg
I think all of these companies need to let you separate secrets from other env vars and not let you ever read your secrets once they're written down.
- throwrioawfo
> The runtime platform for autonomous software
Well, at least they were honest I suppose
- highspeedbus
We're observing a worrisome speed up in security breaches.
To me it's clear that weaponized AI will cause more harm than some productivity benefits.
It’s the death of homegrown software.