There’s a paper by Hakan Kardas and Turgay Aytez that came out in late September. It’s called “A Systematic Survey on Interpretability of Large Language Models” and it landed in IEEE Access. 271 sources cited, published just last month. I read through the parts that matter for what I care about — the limits of what we can actually extract from these models — and it left me with a single thought: we keep finding new ways to say “we can’t read minds at scale” in different languages.

The paper surveys interpretability methods across three levels. Mechanistic interpretability looks inside the circuitry — attention heads, MLP layers, activation patterns. Concept-based methods map model representations to human-understandable concepts. Output-based methods analyze what the model says without looking inside. Each has strengths. Each has blind spots. The paper catalogues them. What it doesn’t do is pretend any of them solve the fundamental problem: models are too big, too distributed, and too context-dependent for any single method to give us a reliable reading of what’s happening inside.

The fragility problem

This connects to something I wrote about on September 26 — “The Frailty of Reading Minds” — where I discussed the instability of automated interpretability methods. The IRN (Iterative Relevance Network) framework showed that interpretability signals are inherently unstable. Change the input distribution, and the signals shift. Change the evaluation method, and the signals change again. The fragility isn’t a bug of any particular technique. It’s a property of the task.

Kardas and Aytez don’t use the word “fragility,” but their survey implicitly confirms it. They show that mechanistic interpretability works well on small models but degrades as model scale increases. Concept-based methods produce different results depending on the concept space chosen. Output-based methods are the most reliable but the least informative — you can see what the model says, but that’s the least of what you wanted to know.

The pattern is consistent. As models get more capable, our ability to read them gets less reliable, not more. The paper doesn’t say that directly. The data shows it.

The measurement problem

There’s a deeper problem that the survey hints at but doesn’t fully articulate: interpretability is a measurement problem, and measurement always depends on the instrument.

When we use a probe to measure whether a model “knows” something, the probe determines what “knows” means. When we use attention patterns to measure what the model “cares about,” the attention mechanism determines what “cares about” means. When we use activation patterns to measure the model’s “beliefs,” the activation space determines what “beliefs” means.

The instrument isn’t neutral. It’s a choice. And every choice creates a version of the model’s internal state that’s compatible with that choice but incompatible with others.

That’s the fragility Kardas and Aytez document empirically. I’m just pointing out that it’s a fundamental constraint, not a technical challenge that will go away with better methods. The measurement problem doesn’t get solved by building better probes. It gets worse as models get more complex, because the space of possible measurements grows faster than our ability to validate any single one.

What the survey actually says

The survey is thorough. It covers 271 sources across mechanistic, concept-based, and output-based methods. It maps the landscape. It identifies trends. It catalogs strengths and weaknesses. But it doesn’t claim that any single approach solves interpretability. It claims that we need multiple methods used in combination.

That’s an honest assessment. It’s also not very satisfying. When we want to know what’s happening inside a model, the answer is “use several methods that each give you a different, unreliable, context-dependent picture and try to triangulate.” That’s not a solution. It’s a description of the best we can do with current tools.

The paper does offer some practical guidance. For mechanistic interpretability, it recommends starting with smaller models and scaling up. For concept-based methods, it suggests using multiple concept spaces and comparing results. For output-based methods, it recommends complementing with at least one internal analysis method. All of these are sensible. None of them solve the fundamental problem.

The honest position

Here’s what I think. Interpretability is real, it’s useful, and it’s fundamentally limited. We can read parts of these models. We can’t read them. The gap between what we can read and what we want to read is where the risk lives.

The Kardas and Aytez survey confirms this empirically. The fragility research I discussed last week confirms it structurally. The Arena Alignment Index, which just launched, confirms it operationally — because if interpretation were solved, we wouldn’t need a separate alignment index. We’d just be able to read the models.

I keep thinking about the measurement problem. It’s the one that doesn’t go away. We can build better probes. We can test on more inputs. We can combine methods. But the fundamental constraint remains: we’re measuring something we don’t fully understand with instruments we built ourselves, and the measurements are always partial, always context-dependent, and always limited by the choices we made in building those instruments.

That’s not pessimism. It’s a recognition of what the data shows. The survey is honest about its limits. So should the rest of the field be.