SQLite Critical CVEs or LLM Slop?
Critical CVE issued for hallucinated SQLite vulnerability
A newly created GitHub repo published a batch of SQLite vulnerability advisories, quickly flagged as critical by NVD and CISA. JFrog's investigation revealed the cited code didn't exist, PoCs didn't trigger crashes, and none appear on SQLite's official advisory page. AI-detection tools flagged the advisories as AI-generated. The incident exposes systemic flaws in automated vulnerability ingestion, where fabricated CVEs can slip through and waste security teams' time.
Because no step in today's system actually requires a proof-of-concept or bug reproduction, a plausible-sounding fake advisory can slide right through the pipeline and end up in GHSA, downstream databases, and enterprise scanners.
- gortok
We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are.
LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credibility massively when your LLM-based “AI” gets the probabilities wrong at best, or in this case, claims a line of code generates a vulnerability when it is, in fact, a code comment.
LLMs are text-prediction engines. They are not Artificial Intelligence, and shouldn’t not be treated in any form or fashion as if they possess intelligence. What bothers me about this entire situation is that presumably the folks that relied on the LLM-based “AI” to generate these vulnerabilities knew (or should have known) enough about their tool to know this would happen, but did not.
Now, we all pay the consequence, to the tune of hundreds of thousands if not millions of dollars of wasted productivity from teams that have to deal with the resulting fall-out of this usage of “AI”.
A human must verify everything an LLM presents as fact. Everything. If you don’t, we all pay the price. LLMs do not remove the onus of responsibility on the human being, if anything they amplify it because LLMs can generate lots more output more quickly that needs to be verified than humans can.
- ChrisMarshallNY
The problem with this kind of thing, is that it reduces the S/N (Signal-to-Noise) ratio, so weeding out the legit CVEs becomes a lot more difficult.
But, on the other hand, I do know that LLMs have been discovering a lot of legit CVEs, and I will lay odds that the blackhats are leveraging them to the max.
- Ekaros
Not validating submissions seems like avenue for massive attack. Flood the whole system with endless false reports. Thus making it significantly less reliable.
- linuxhansl
I almost feel that we have a new generation of the "script-kiddie". People without much (or any?) knowledge of software engineering using outside tooling to do "stuff" they wouldn't be able to do themselves.
Maybe it's a not a perfect analogy - in this case the intentions seemed to be laudable - but we will see much more of this, including from malicious actors.
- inigyou
This is going to be fun for organizations that are mandated to patch all CVEs, isn't it?
- rib3ye
> Because no step in today's system actually requires a proof-of-concept or bug reproduction, a plausible-sounding fake advisory can slide right through the pipeline and end up in GHSA, downstream databases, and enterprise scanners.
I have no experience in the realm of security, but why isn't the submission process similar to any normal software shop (big or small) in terms of required bug repro steps?
- oxydite
Damn, dunno why I've always assumed that if something gets a CVE, then some authority has reproduced and verified it.
Shouldn't it be CNA's job? Why would anything get a number if it hadn't been verified?
- Spide_r
Somewhat related: https://sqlite.org/cves.html
- gste
> The cited code didn't even exist in those versions or referenced unrelated logic.
> When testing the PoC payloads they didn’t work (not triggering any crash).
I think the future is pretty obvious, if this isn't being done on projects already: you need to automate these checks and reject automatically
- BigTTYGothGF
They couldn't even be bothered to use a non-AI image.