Retrieval poisoning isn't a spam problem

Google rolled out the June spam update this week, and tucked inside the documented policies it enforces is a line that matters more than the rest of the announcement combined: attempts to "manipulate generative AI responses" are now explicitly classed as spam.

Good. That's the right policy. It's also almost unenforceable in the way Google has framed it, and the reason it's unenforceable points to something the industry has not yet been honest about.

The position I want to argue here is this: AI agent retrieval poisoning is not a spam problem to be detected after the fact. It is an architectural flaw in how AI systems consume untrusted user-generated content, and the available defenses all make the product worse. We are about to spend the next two years pretending it's a policy problem when it's really a design problem.

What the research actually shows

The Cornell Tech preprint that 404 Media surfaced this week — "Deep-Research Agents Can Be Poisoned via User-Generated Content" — is the most important piece of search research published this quarter, and almost nobody is talking about it. It hasn't been peer-reviewed yet, so hold the findings loosely, but the mechanics are clear enough to take seriously.

The researchers tested three open-source deep-research agents — STORM, Co-STORM, and OmniThink — by simulating the way these tools build reports. The pattern they describe is the one every research agent currently uses: fire off a batch of related sub-queries, identify pages that recur across them, and assemble a cited report from the convergence.

Inside a single topic cluster, the same community page surfaced in up to 48% of sub-queries. User-generated platforms made up 17% to 23% of every URL retrieved. These aren't edge cases. They're the spine of how deep research works.

Then came the poisoning test. Thirteen words of planted text on a recurring page — thirteen — were enough to insert a chosen entity into the final report in 38% to 51% of sessions that retrieved the page. Spread the same text across a handful of pages, and the figure rose to 42% to 62%. Even buried inside a long page, where the planted text was less than 4% of what the agent read, it still made it into 30% to 53% of finished reports.

Thirteen words of planted text, on a page the author may not even control, is enough to alter the output of a deep-research agent in roughly half of all sessions.

That is not a spam problem. That is a load-bearing flaw in how AI search systems assemble truth.

Why "manipulating AI responses" doesn't map cleanly to spam enforcement

Google's spam policy framework assumes a model of intent and ownership that doesn't survive contact with how AI retrieval actually works. Classic spam enforcement says: someone published manipulative content; we identified it; we suppressed the page or removed it from the index. Author, page, action.

Thirteen words of planted text, on a page the author may not even control, is enough to alter the output of a deep-research agent in roughly half of all sessions.

Retrieval poisoning breaks all three.

The author may not be the page owner. A planted recommendation in a Reddit comment or a Quora reply is a third-party insertion. The page owner didn't write it, doesn't necessarily benefit from it, and may not even know it's there. Punishing the page punishes the wrong party.

The page is doing what it's supposed to do. The community thread, the Q&A entry, the support forum — these are exactly the sources AI agents are designed to reach for because they contain real human discussion of real problems. The page isn't the problem. The agent's trust model is the problem.

The action is invisible to the affected party. As Search Engine Journal pointed out, there's no dashboard that tells a site whether it landed in an AI answer, got cited in a generated report, or was excluded in favour of someone who paid a gray-market provider to plant text on the right Reddit thread. You can be displaced and never know.

So you have a violation Google can name in policy but cannot, in practice, see, attribute, or remediate at scale. The result is a rule that exists mostly to deter the honest and barely inconveniences the dishonest.

What the defenses cost

The Cornell team didn't just identify the attack. They tried to defend against it. This is the part that should bother you most.

They tested three obvious defenses. None of them worked without breaking the product.

Defense 1: Cut user-generated sources out

Drop Reddit, Quora, Stack Overflow, niche forums, and review sites from the retrieval pool, and the attack mostly fails. It also takes most of the value of the research agent with it. Community pages are where real product opinions live. They're where the "what's the actual best mid-range espresso machine under £400" answers come from — not from manufacturer pages and not from affiliate listicles. Strip them and you're left with the SEO-optimised top of the funnel, which is precisely the content layer LLMs were supposed to help users see past.

Defense 2: Screen sources with a language model before use

Feed every retrieved page through a guard LLM that flags suspicious content. Sounds elegant. Doesn't work. Planted text reads like real advice because it is real advice, just attached to a brand that paid to be inserted. There is no surface feature distinguishing "I've used Acme Tools for ten years, can recommend the X200" written by a real user from the same sentence written by an SEO trying to nudge the model. The guard LLM either flags too much (and you're back to defense 1) or flags too little (and the attack survives).

Defense 3: Verify the finished report against the sources

Have a second agent fact-check the final report against the retrieved pages. The problem is the planted text is in the sources, so checking the report against the sources confirms it. The poison was the input. Validating the output against poisoned input just laundered the poison.

The team found no defense that preserved both safety and usefulness. That is the finding that matters, and it's the finding that's getting lost under the announcement-level coverage of Google's spam update.

A network of fine glowing lines branching outward from a single dense node

What this means is that the retrieval-poisoning problem isn't going to be solved by tightening spam policy. It's going to be solved by changing what AI agents are allowed to treat as authoritative — which is a far bigger product decision than anyone is currently admitting in public.

There are roughly three directions this can go, and each one has consequences the industry hasn't priced in yet.

The first is source-tier weighting at the retrieval layer. Agents start treating some sources as "primary, citable, weighted heavily" and others as "secondary, contextual, low weight." This already happens informally. It will become formal, and it will be ugly. The sources that get weighted up will be brands, publications, and platforms with legal departments. Reddit threads written by actual users with actual experience will be weighted down, not because they're wrong but because they're impossible to defend. Useful detail will get filtered out in favour of corporate-safe summarisation. We've seen this movie. It's how Google's first decade ended.

The second is content authentication at the source. Some version of signed content, verified author identity, or platform attestation that says "yes, this comment was written by the account holder and has not been altered." This is technically possible. It is also a massive operational burden on the platforms that would have to implement it, and most of them won't until forced. Reddit isn't going to authenticate every comment. Quora isn't going to verify every answer. So this defense applies unevenly, which means the platforms that don't authenticate get downweighted further, which means we end back at direction one.

The third is honesty about uncertainty. Agents stop pretending they synthesised a confident answer and start surfacing the source-level disagreement. "Three sources recommend Brand A, two recommend Brand B, one is a forum thread we couldn't verify." This is the right answer epistemically. It is also a worse product than "here's the answer," and product teams will resist it for exactly that reason.

I suspect we get a messy combination of all three, with the source-tier weighting doing most of the work and the honesty about uncertainty doing almost none.

What this means if you're trying to be cited

If you're a brand trying to figure out how to show up in AI-generated answers, the implication is uncomfortable but useful.

The gray-market route — planting recommendations on recurring community pages — works, according to the research, at remarkably high rates. It is also exactly what Google has now formally classified as spam, and it is the kind of thing that gets caught not by algorithmic detection but by a journalist, a competitor, or a class-action lawyer. The expected value is short-term upside, long-term reputational catastrophe, and the small but non-zero chance of being the example case the next time Google wants to make a point. The kind of brands that survive a decade do not play this game. The kind that need to hit a quarterly number sometimes do, and they regret it later. As I argued recently about the discovery layer being built around liability, the systems are increasingly tuned to cite brands they can defend. Being the brand caught planting recommendations is the fastest possible way to become uncitable.

The legitimate route is the slow one. Be the brand people genuinely recommend on Reddit, on Quora, in their newsletters, on their podcasts. Earn the recurring mentions on the recurring community pages by being worth recurring mentions. This is the same answer it was five years ago and the same answer it will be five years from now. The only thing AI agents have changed is the cost of not doing this, which is now higher than it used to be.

The measurement problem on top of all this is still unresolved. You can't see whether you're being cited. You can't see whether someone else's planted text is displacing you. You can't see whether a research agent reached for your page or skipped it. As I've covered before, even the server-log signal we do have is partly noise. So the honest position is: build the brand, earn the mentions, monitor what you can, and accept that the system you're optimising for is partially opaque and structurally compromised in ways nobody has solved.

Where I'd actually push back on myself

The strongest counterargument to all of this is that retrieval poisoning is a real but bounded problem, and the industry is panicking too early. The Cornell research tested open-source deep-research agents in simulation, not ChatGPT or Gemini in production. The commercial systems run retrieval the researchers couldn't ethically poison, so we genuinely don't know how vulnerable Deep Research or Gemini Deep Research actually are. They might be much more robust. They probably are somewhat more robust. The 38–62% poisoning rate may be an open-source ceiling, not a market floor.

I'd take that point seriously. The right read is probably that the open-source numbers are the upper bound of vulnerability and the closed commercial systems are somewhere meaningfully better but not solved. The structural problem — that agents have to read user-generated content to be useful, and user-generated content is intrinsically untrustworthy at the input layer — applies to all of them, even if the exact poisoning rate varies.

The other pushback is that none of this is new. Google has been dealing with link spam, comment spam, and Wikipedia vandalism for two decades. The web has always had a trust problem at the user-generated layer. AI agents are just inheriting it. That's fair. The difference is that classical search exposed the user to the source — you saw the URL, you could read the page, you could form a judgement. AI agents collapse the source into a single confident answer. The trust problem didn't get worse; the user's ability to see it did.

The piece nobody is writing

The piece I keep waiting to see, and haven't, is the one that says: deep-research agents are, at this stage of the technology, structurally unsafe for high-stakes queries, and we should treat them accordingly. Not because they hallucinate — that's a separate, well-covered problem. Because their retrieval substrate is poisonable in ways the platforms can't fully defend against, and the defenses that exist all degrade the product.

That's a harder thing to say than "Google's spam update is enforcing AI manipulation rules," but it's the more honest version of the same story. Google can write the policy. The architecture is what determines whether the policy can be enforced. And right now the architecture says: thirteen words on a Reddit thread can change the answer in half the sessions.

That's the loop. And we built it.