Anthropic published four weeks of optimization research. We turned it into a memory an agent could build on.

Here's an optimization an AI coding agent gave me for a protein-folding model called OmegaFold:

"Fuse the pairwise distance and angle projections into a single kernel launch, since both read from the same intermediate tensor and currently trigger separate memory round-trips."

It sounds like a real, specific, technically grounded suggestion. It's also fake. I wrote it myself for this post to make a point: no agent produced it, it isn't in any real codebase, and as far as I know it wouldn't even work. Read next to a real suggestion, though, it sounds exactly as confident and exactly as specific.

That's the actual problem this post is about. An AI agent can sound certain and precise while being completely made up, and reading the sentence alone won't tell you which one you're looking at. The only thing that can is being able to trace the claim back to where it actually came from. That's what the rest of this post tests, using Anthropic's own published research as the material.

What Anthropic actually did

Anthropic put out a report last week. Over four weeks, they had Claude optimize more than 30 open-source biology models, things like protein structure predictors and protein design tools, with two of their own scientists supervising who'd never done this kind of low-level performance work before.

Claude made these models faster in two different ways. "Fast mode" traded away a little numerical precision for speed and averaged 4.1x faster. "Exact mode" kept the output bit-for-bit identical to the original and still averaged 1.6x faster. Anthropic published all of it, one folder per model, each with real notes explaining exactly what was changed and why, in a public repo called uplifting-biomolecular-modeling.

That's four weeks of two scientists' real, checkable expert work, sitting in public for anyone to read.

What I tested

Empirical is a memory tool: you load real documents into it once, and from then on, any AI agent you're using can search that memory before it answers, instead of only working from what it already knows. What I wanted to know was simple: does giving an agent real prior work to search actually change what it comes back with, or is it just extra reading the agent ignores?

To test it, I loaded all 36 of Anthropic's optimization write-ups into Empirical as a searchable memory, one entry per model. That took about fifteen minutes.

Then I picked a real model outside Anthropic's study: OmegaFold, a protein structure predictor built by a different team entirely. Its own documentation admits a real weak spot, past a couple thousand amino acids it runs out of GPU memory, and you have to hand-tune a setting to work around it.

I gave two AI agents the identical task: read OmegaFold's real code, and come back with eight specific ways to make it faster or use less memory.

• Agent A could search the 36 Anthropic write-ups in Empirical before answering.
• Agent B had no search at all, just whatever it already knew.

Everything else about the task, the model, the instructions, the number of suggestions asked for, was the same for both.

What each agent came back with

• Agent B gave eight suggestions that were all reasonable and none of them wrong exactly, things like using a lower-precision number format or skipping some default bookkeeping code. Standard advice. None of it was tied to OmegaFold specifically, and none of it pointed at any prior work, real or otherwise.
• Agent A also gave eight suggestions. I checked every single one by hand against Anthropic's actual GitHub files to see where it really came from. Seven traced straight back to a specific Anthropic write-up, the same technique, described the same way. The eighth wasn't in Anthropic's material at all: it turned out OmegaFold recomputes a piece of its own math on every pass that never actually changes between passes, something specific to how OmegaFold itself is built, not something any of Anthropic's 36 models needed to fix.

Grounding the agent in real prior work didn't make it parrot sources back at me. It changed what it noticed.

Once the agent had real examples of what "an optimization" looks like in this exact domain, reasoning from OmegaFold's own code to spot something nobody had written down got easier, not harder. A blank page doesn't do that.

Checking the receipts

Here's one of the seven, in full, so you can see what "traced back" actually means. Agent A proposed: record the GPU's repeated steps once, then replay that recording instead of reissuing the same instructions on every single pass.

That's exactly what Anthropic's team did for one of their own models, ESMFold2. Their internal notes give every individual optimization a short two-letter code name (there are dozens across the file, ec, t3, xte, and so on) so ls and rg below are just Anthropic's own labels for this one, not commands:

esmfold2/CHANGES.md
`ls`, `rg` (Fast) — the recycle loop with loop-resident static I/O;
one CUDA graph per recycle, replayed for the fold's remaining recycles

You can read the real file yourself. That's the whole idea of "checking the receipts": not "trust me," a link you can click.

The other six proposals traced the same way, to Anthropic's AF3, OpenFold3, and Chai-1 write-ups:

That's what "specific" is supposed to buy you: a claim you can walk back to a source and check yourself. My opening example had the same tone, the same confidence, the same plausible mechanism, and none of that. The only thing that tells the two apart is provenance, not how confident either one sounds.

Loading the memory, running both agents, and checking every proposal by hand against Anthropic's real files took under two hours total, against four weeks of hands-on work from two of Anthropic's own scientists.

Why this isn't just "search helps"

Fair question: couldn't you get the same result from a plain search tool, or by just pasting all 36 files into the agent's context window for one conversation? For this one task, probably. What's different is what happens after, and whether you can trust the answer later.

I loaded those 36 files into Empirical once. They're available to every future conversation, in whatever AI tool I open next, without me rebuilding anything or pasting the files in again. A one-off script or a big pasted context gives you that search for one conversation, then it's gone, and nothing about it tells you which answers to trust.

Empirical also keeps the link from each fact back to where it came from. When Agent A pulled a technique from Anthropic's ESMFold2 write-up, that link is saved alongside it. If one of those seven proposals had turned out to be wrong, everything built on top of it would be easy to find and remove, not just that one claim. I tested that direction a few weeks ago on a different project: planted one bad fact on purpose, and it, and everything downstream of it, was easy to trace and pull out. A pasted context window can't do that after the fact, because nothing kept the receipt.

And it doesn't stop at Anthropic's 36. The OmegaFold optimization Agent A found on its own didn't come from their write-ups, it came from reasoning against them, and that new finding went back into the same memory the same way Anthropic's originals did: it's a real, shipped check in the Inference Optimization Brain blueprint below, not just a line in this post. Run this against ten more models outside Anthropic's study and the memory a future agent searches isn't 36 entries anymore, it's 36 plus whatever real, checked techniques got fed back in. Anthropic's four weeks was the seed, not the ceiling.

Try it

Your own documentation, code comments, and past incident write-ups are the same kind of material Anthropic's notes are: real, specific, and traceable. Loaded into Empirical, an agent can reason from them the same way Agent A did here, including finding the thing nobody wrote down, the way it did with OmegaFold.

I built the exact memory from this post so you can run your own version of it. I checked all 37 entries by hand against Anthropic's real files.

Once it's installed, paste this to whatever AI agent you connect to Empirical with:

I just installed Inference Optimization Brain. Search my memory for optimization
techniques relevant to [YOUR MODEL], then propose 5-8 specific ways to speed it up
or cut its GPU memory use. For each one, name the real technique you found and say
which mode it belongs in: exact (bit-identical output), fast (small precision
tradeoff for speed), or big (less memory, larger inputs).

Swap in whatever model you're actually trying to optimize. A real answer names its source. If it doesn't, that's the tell, the same tell that gives away the sentence at the top of this post.

All Empirical blog posts