We reran part of Anthropic's biology experiment. The agent with memory won.
Anthropic had Claude optimize 30-plus open-source biology models over four weeks. I gave two agents a similar task on a model outside that study, one with Empirical, one without. Same task, same facts, one real difference.
Anthropic put out a report last week. Over four weeks, Claude optimized more than 30 open-source biology models, structure predictors, protein design tools, that kind of thing, run by two of their own scientists who'd never done that kind of optimization work before. The results are real: their fast mode averaged 4.1x faster, their exact mode (bit-for-bit identical output) averaged 1.6x faster. They also open-sourced all the optimized code and used the same work to kick off a protein design competition with Adaptyv Bio.
We recreated the core of it in Empirical, just as effective and a lot quicker to set up. Try it yourself below.
The report measures every model's speed in careful detail, but it never checks whether solving one model made the next one any easier for the agent to solve. That's the part I actually cared about, because it's the same question I ask about Empirical every day: does saving what an agent already figured out make its next answer better, or does it just start over each time? Anthropic's report gave me something I don't usually get: a large, real, publicly checkable body of expert work I could load into Empirical and test that question against, instead of a toy example.
I couldn't run Anthropic's full study. I could run a small, honest version of it on my own project, using their own published work as the material. Here's the shape of it:
The setup
Anthropic put all 36 of those optimization packages on GitHub, in uplifting-biomolecular-modeling, each with real documentation explaining exactly what it changed and why. I loaded that whole set into Empirical, one entry per model, as a corpus an agent could search.
Then I picked a real model that isn't in Anthropic's study: OmegaFold, a single-sequence protein structure predictor from a different team. Its own documentation admits a weak spot: past a couple thousand residues it runs out of GPU memory, and you have to hand-tune a setting to work around it.
I gave two agents the same task: read OmegaFold's real design and propose specific ways to make it faster or use less memory. One agent could search the corpus. One had no search at all, just what it already knew. Everything else about the task was identical.
What each one actually produced
The agent with no search gave eight reasonable, correct suggestions. Things like using a lower-precision number format, or skipping some bookkeeping PyTorch does by default. Standard advice. None of it tied to anything specific about OmegaFold or to any prior work.
The agent with Empirical also gave eight proposals, and every one of them pointed at something real:
| What it proposed, in plain terms | Where the idea actually came from |
|---|---|
| Record the model's repeated GPU steps once and replay them, instead of reissuing the same instructions on every pass | Anthropic's ESMFold2 package does exactly this |
| Store the model's weights in a smaller, faster number format at load time | Anthropic's AF3 package does this |
| Skip loading and immediately overwriting the model's starting weights before its real weights are read in | Anthropic's OpenFold3 package does this |
| Let the software pick the memory-saving batch size automatically instead of making the user guess it by hand | The same automatic approach Anthropic's Chai-1 package uses for its own memory limit |
I checked all eight, by hand, against Anthropic's actual GitHub repository. All eight were real. Not paraphrased, not close enough, actually the same technique described the same way in Anthropic's own files. Here's the actual line behind the first one, from Anthropic's own file for their ESMFold2 package:
esmfold2/CHANGES.md
`ls`, `rg` (Fast) — the recycle loop with loop-resident static I/O;
one CUDA graph per recycle, replayed for the fold's remaining recyclesIn plain terms: record the GPU's repeated steps once, then replay that recording instead of reissuing the same instructions every single time. That's the exact technique the agent proposed for OmegaFold, and this is the real file proving Anthropic's own team used it too.
One of the eight wasn't in the corpus at all. OmegaFold recomputes a piece of its own math on every pass that never actually changes between passes, something specific to how OmegaFold itself is built, not something any of Anthropic's 36 packages needed to solve. The agent found that on its own, once it had real prior work to reason from instead of a blank page.
Why the agent with Empirical did better
Both agents were equally capable. The only thing that changed between them was whether one could look something up before answering.
Fair question: couldn't you get the same search with a plain RAG setup, or by just pasting those 36 files into a long context window? For this one task, probably yes. The part that's actually different is what happens after. I loaded those 36 files into Empirical once. They're available to every future session, in whatever tool I open next, Claude, ChatGPT, the CLI, without me rebuilding anything or pasting anything again. A one-off script or a big context window gives you the search for that one conversation, then it's gone.
It also keeps a record of where each fact came from. When the agent pulled a technique from Anthropic's ESMFold2 package, that link is saved with it, so if one of those eight proposals ever turned out to be wrong, it would be easy to find everything built on top of it and fix that too. I checked that direction a few weeks ago on a different project: planted one bad fact on purpose, and it and everything downstream of it were easy to find and pull out.
Try it
Anthropic's real published work is what made the difference here, and it's public. Your own documentation, code comments, and past incident writeups are exactly the same kind of material, and Empirical loads them the same way. Give an agent something real to search, and eight honest guesses can turn into eight real answers.
Every one of these 37 is the same corpus, ready to install into your own account. I checked all of them by hand against Anthropic's real files, go ahead and try to find the one that doesn't hold up.
Once it's installed, paste this at whatever agent you're connected to Empirical with:
I just installed Inference Optimization Brain. Search my memory for optimization
techniques relevant to [YOUR MODEL], then propose 5-8 specific ways to speed it up
or cut its GPU memory use. For each one, name the real technique you found and say
which mode it belongs in: exact (bit-identical output), fast (small precision
tradeoff for speed), or big (less memory, larger inputs).Swap in whatever model you're actually trying to optimize. If the answer comes back with real named sources instead of generic advice, that's the corpus doing its job.
Eight for eight isn't luck.
record_graph_memory loads whatever your team already knows into a corpus any connected agent can search with query_memories. Every fact keeps a link back to where it came from, so the next agent isn't guessing either.