A third of Perplexity's citations don't contain the number they're cited for
An audit of Perplexity's search models reveals that 34.7% of citations attached to numeric claims point to pages that are inaccessible or lack the cited figures. Testing 310 factual questions about tech companies, the study found that 14.4% of claims fail entirely. The failures stem not from dead links (only 1.3%) but from paywalled pages and pages that simply don't support the claim. The report highlights a systemic issue: AI citations often point to pages that never contained the information, undermining the purpose of citation.
The claim and the citation were produced by the same process, and that process is not retrieval.
- breakpointalpha
This article appears to be a coordinated attack on Perplexity.
Also posted today is this HN thread:
https://news.ycombinator.com/item?id=49536375
Containing this link:
https://trellner.com/reports/manufactured-sources-behind-ai-...
Which has the exact same layout and very similar About page as "trellner.com"
- nightpool
I don't even know if I disagree with this post, but this seems really astroturfed. Why are there two anti-Perplexity articles from independent research firms with identical websites on the front-page of HN right now, submitted by the same person? Am I going crazy?
(see https://news.ycombinator.com/item?id=49536375, left a comment there also)
- wrs
Same is true of Google search “summaries” where quite often I click on the link provided and it doesn’t support the statement Google made.
BTW, this post would be more convincing if it wasn’t written in Claude voice itself!
- abdullahkhalids
If you are building your own harness that does correct citations, is the correct thing to give AI access to some deterministic tool that allows it to actually copy paste parts of documents its reading (with links), rather than stochastic reproduction that they do by default?
- Gecko4072
I noticed this personally. Saw a citation with a preview for source A, which I knew was reliable. Checked, and it referenced a Reddit article and various other less reliable sources. Was a direct citation too that actually wasn't.