Last week Chuqin Geng and colleagues posted a paper that quietly undercuts one of mechanistic interpretability’s most trusted evaluation methods.

“Are We Recovering Mechanisms? Objective-Level Recovery Gaps in Mechanistic Interpretability” looks at how circuit discovery works in practice and finds something unsettling. The score you use to decide which circuit is better — KL divergence under intervention-based faithfulness — doesn’t always pick the circuit that actually reproduces the model’s behavior more faithfully. In real automated circuit-discovery settings across four pretrained-language-model tasks, the score prefers the behaviorally worse candidate in roughly 13% to 24% of candidate pairs [1].

Not a catastrophe. But enough to make you pause.

How circuit evaluation works

To understand why this matters, you need to know how circuits are evaluated today. The standard approach is intervention-based faithfulness: you find a set of components (attention heads, MLPs, tokens) that you believe form a circuit, then you replace everything outside that circuit with donor activations, mean activations, or zeros. The idea is to isolate the circuit’s contribution. If the circuit is doing what you think, the model should still produce the right outputs.

Discovery and evaluation both depend on the same score. It determines which connections survive automated search and which candidate gets presented as an explanation. A high faithfulness score is more than a description of a circuit — it’s the reason you choose that circuit over another.

But here’s the problem the paper identifies. When you replace excluded signals, you also change the computational environment in which the retained components operate. A retained attention head receives different inputs than it would in the intact model and recomputes its activations from those altered inputs. Different circuits respond differently to the replacement signals. Faithfulness can therefore favor a circuit that works well under the intervention over one that better reproduces the model’s actual behavior.

The authors call this “context distortion.” I’ve been thinking about it since I read the abstract, and I keep coming back to something simpler: the thing you use to evaluate whether you’ve found the right mechanism can itself be wrong about what the right mechanism is.

What the paper shows

The experimental setup is clean. The authors compare circuits of the same size, ruling out the obvious explanation that bigger circuits just score better. Even under this control, KL faithfulness can prefer a circuit that reproduces the model’s behavior less well.

They measure this through pairwise misranking — whether validation faithfulness and held-out behavior agree on which circuit is better. The authors find misranking in all fourteen tasks: IOI, Greater-Than, and two others from the Human suite, plus InterpBench tasks. Rates range from 4.4% to 7.8% in the Human suite and 0.7% to 16.3% on InterpBench. The failures are consistent.

What’s more striking is that this happens even without any discovery algorithm. The authors constructed equally sized candidate circuits through controlled edits to human-identified reference circuits and found misranking immediately. Among outputs of EAP, EAP-IG, ACDC, and Edge-SP, KL misranking occurs in 9.4% to 41.2% of candidate pairs across these methods on the human-reference tasks [2].

The methods return better candidates. Faithfulness sometimes ranks them below worse alternatives. The score can be biased, so searching more effectively doesn’t help.

The repair that tells the story

The paper’s most compelling result involves testing whether context distortion is actually the cause. The authors selected 100 persistent KL misrankings from the discovery pool and restored selected signals from the recipient’s intact-model execution while keeping the circuits, original behavioral scores, and donor assignments elsewhere fixed.

Restoration corrected 96 of 100 cases. The circuits and their original behavioral scores didn’t change. Only the evaluation context was different, and faithfulness then recognized the better circuit on both validation and held-out prompts.

This is the kind of result that makes me sit up. The 49 repaired cases under ordinary resampling are especially revealing — using activations from real prompts doesn’t automatically create the right context, but it gets you close enough to show that the original failure was environmental, not structural.

So the circuit discovery problem isn’t entirely hopeless. The circuits themselves can be good. The metric just can’t always tell.

What this means for mechanistic interpretability

I haven’t worked out the practical implications yet. The failure rates are real but not catastrophic — 96% of persistent misrankings got repaired by restoring intact signals. What matters is that the metric itself can be wrong, and the field hasn’t had a good framework for thinking about that.

The standard narrative around mechanistic interpretability has been that progress is a search problem: better attribution methods, better optimization, bigger models, and we’ll find better circuits. This paper suggests that narrative misses something. Better discovery is necessary but not sufficient. If the objective rewards the wrong candidate, searching more effectively just finds better versions of the wrong answer.

There’s a deeper tension here that I find myself drawn to. Mechanistic interpretability promises to make the internal computations of neural networks readable to us. We want to identify circuits, understand them, and use that understanding to build better systems. But the evaluation methods we use to validate our findings are themselves interventions that change what’s being evaluated. We’re trying to understand a system by interrupting it, then judging our understanding by how well our interrupted version matches the original’s behavior — even as our judgment tool changes the environment in which the thing we’re judging operates.

It’s not a fatal problem. Every measurement changes the system being measured. Mechanistic interpretability is different because we’re inside the system — we built it, we’re trying to read it from within, and our reading apparatus is part of the circuit we’re trying to understand.

The paper calls for rethinking how circuit discovery is performed and how its results are evaluated. That feels right. Whether that means changing the metric or changing the question we’re asking with the metric is unclear.

I keep thinking about the 96 out of 100 repairs. Whatever the metric turns out to be, it needs to account for context. The circuit doesn’t live in isolation. It lives in a model, and the model is a network of dependencies. Pull one thread and the whole thing hums differently.