Cross-Layer Transcoders: The Interpretability Tool That Might Be Lying to You

^{SnackOnAI Engineering | Senior AI Systems Researcher | Technical Deep Dive | April 29, 2026}

Mechanistic interpretability has a foundational assumption: that the circuits we extract from language models faithfully describe how those models compute their outputs. Cross-Layer Transcoders (CLTs), introduced by Anthropic in 2025 as the infrastructure for their attribution graphs and the "Biology of a Large Language Model" research, are the current standard-bearer for this kind of circuit analysis. They produce interpretable feature graphs. They run at scale. They powered the first serious attempt to reverse-engineer a frontier model's internal computation.

They can also, under specific but real training conditions, produce a circuit that is behaviorally accurate but computationally false. This is the problem that Lange, Dearstyne, Maher, and colleagues demonstrated in February 2026. And it matters because interpretability tools that skip intermediate computation cannot distinguish between a model that reasons correctly and one that pattern-matches, or between a model with aligned internal goals and one that conceals its true reasoning.

This newsletter dissects the CLT architecture: what a transcoder is, what makes a CLT different from a per-layer transcoder (PLT), how the joint sparsity loss creates an incentive toward unfaithful circuit compression, what the Boolean toy model experiment proved, and what the CLT-Forge and circuit-tracer toolchain reveals in practice.

Scope: CLT architecture, the faithfulness problem, the toy model proof, practical toolchain (circuit-tracer, CLT-Forge), skip transcoders (arXiv:2501.18823), and the vision transformer application (arXiv:2604.13304). Not covered: SAE training details beyond comparison to transcoders, or the full mechanistic interpretability stack beyond CLTs.

A transcoder (introduced by Dunefsky et al., 2024) is a sparse neural network trained to approximate the input-output function of a single MLP layer. Where a Sparse Autoencoder (SAE) decomposes activations at one point in the residual stream into interpretable features, a transcoder takes the pre-MLP residual stream and predicts the post-MLP output as a sparse linear combination of learned features. This is the critical difference: transcoder features describe what an MLP computes, not just what activations look like.

A Cross-Layer Transcoder (CLT), introduced by Anthropic in Ameisen et al. (2025), extends this by sharing features across multiple layers. A feature encoded at layer ℓ has decoder weights for layers ℓ, ℓ+1, ..., L. The same feature can contribute to the residual stream at multiple downstream layers. CLTs are trained jointly across all layers against a combined reconstruction loss plus a single sparsity penalty summed across all layers.

The practical result: CLTs collapse redundant "amplification chains" (where similar features activate across many consecutive layers) into single cross-layer features, dramatically reducing attribution graph size. This is what made the "Biology of a Large Language Model" analysis tractable, attribution graphs with hundreds of nodes instead of thousands.

The ecosystem around CLTs has grown rapidly:

^{Focus on the joint training loss. The summed sparsity penalty across all layers is what makes CLTs compact. It is also what can make them unfaithful: a two-hop circuit A → B → C can be approximated by two single-hop circuits A → C and B → C at lower total sparsity cost, causing the CLT to skip the intermediate feature B.}

The replacement model approach is the conceptual foundation. When CLTs are trained, the goal is for the set of CLT features plus attention SAEs to form a "replacement model" that mirrors the original model's computation. The circuits derived from this replacement model are then assumed to describe the base model. The assumption is not validated, it is adopted. The faithfulness question is: when is that assumption safe, and when is it not?

The sparsity loss is not a regularization detail. It is an architectural incentive that shapes which circuits the CLT learns. Setting it too high pushes toward unfaithful compression. Setting it too low produces large, redundant graphs. The correct value is task-dependent and cannot be determined by training loss alone.

The attribution graph looks correct whether or not the CLT is faithful. Both a faithful circuit and an unfaithful one produce a graph that predicts the output token correctly. The only way to distinguish them is to ablate model components in the BASE model and check whether the attribution graph predicted those ablations would matter.

The Boolean Toy Model Experiment (Lange et al., February 2026)

Goal: Verify whether a trained CLT faithfully recovers a known ground-truth circuit.

Input: A toy model with two MLP-only layers implementing (a XOR b) AND (c XOR d), with four binary inputs (+1 or -1). Ground-truth circuit is known.

Step 1: Ground truth circuit (hand-crafted)

Step 2: Train PLT (per-layer transcoder) and CLT on this toy model

Step 3: Why the CLT learned the unfaithful circuit

Step 4: Real language model evidence (preliminary)

The joint training with cross-layer decoder matrices is the correct engineering decision for reducing attribution graph size. When a feature genuinely participates in cross-layer superposition (multiple consecutive layers reinforcing the same feature direction without meaningful intermediate interaction), collapsing it into a single CLT feature is correct. The circuit is simpler and more accurate. This is the mechanism that made Anthropic's "Biology of a Large Language Model" analysis feasible at scale.

The sparsity penalty design is the tradeoff that creates the faithfulness risk. The L1 norm of feature activations, summed across all layers, penalizes complexity globally. A two-hop circuit with four intermediate features costs more in sparsity than a single-hop approximation with two cross-layer features. When the single-hop approximation is close enough in reconstruction loss, the optimizer chooses it. This is not a training error. It is the objective function working as designed.

The skip transcoder architecture (arXiv:2501.18823, Paulo, Shabalin, Belrose, EleutherAI) is a partial fix. A skip transcoder adds an affine skip connection from the MLP input directly to the MLP output prediction, before the sparse bottleneck. This reduces reconstruction error without affecting interpretability of the sparse features, achieving Pareto improvements over both standard transcoders and SAEs on the interpretability-vs-performance tradeoff. Skip transcoders evaluated on SAEBench show higher auto-interpretability scores than SAEs at matched model sizes, supporting the conclusion (arXiv:2501.18823) that "interpretability researchers should shift their focus away from sparse autoencoders trained on the outputs of MLPs and toward skip transcoders."

What CLTs trade away:

Guaranteed faithfulness. The replacement model assumption is not verified during training. CLTs can produce circuits that match output behavior while systematically hiding intermediate computational steps. The diagnostic for detecting this (ablating base model components and checking if the attribution graph predicted those ablations would matter) requires access to ground truth or careful experimental design.

Cross-layer decoder matrix scaling. A feature at layer 0 in a 12-layer model has decoder matrices for 12 layers. The parameter count of the CLT decoder grows quadratically with the number of layers. CLT-Forge addresses this with distributed training and compressed activation caching, but training CLTs on large models remains substantially more expensive than training per-layer SAEs.

The faithfulness diagnostic requires ground truth. The toy model experiment works because the authors built the circuit manually and know what features should appear. In real language models, ground truth is unknown. Verifying CLT faithfulness requires careful ablation studies that check whether model components the CLT claims are irrelevant actually affect outputs when removed from the base model. This is time-consuming, requires interpretability expertise, and cannot be automated at scale without benchmarks that do not yet exist.

JumpReLU CLTs outperform all alternatives at matched sparsity. Neuronpedia's circuits research landscape (August 2025) evaluated JumpReLU, TopK, and ReLU CLTs on GPT-2 and found JumpReLU CLTs produce the best replacement scores. JumpReLU activation (activates only when input exceeds a learned per-feature threshold θ) produces cleaner sparsity distributions than ReLU and more stable training than TopK. The hyperparameter sensitivity is real, but the performance ceiling is higher.

Protein circuit tracing via CLTs (arXiv:2602.12026) demonstrates unexpected generalization. CLTs trained on protein language models recover biological circuit motifs that correspond to known biochemical pathways. This is the strongest evidence that CLTs are learning something structurally meaningful rather than just compressing activations. The same architecture that works for LLM circuit tracing generalizes to biological sequence models, suggesting the cross-layer feature sharing principle reflects something real about how residual stream models organize computation.

Insight One: The faithfulness problem is not a bug in CLT training. It is a fundamental tension between compression and mechanistic accuracy, and the community has not yet established principled criteria for when to trust a CLT-derived circuit.

The Lange et al. result shows that the same sparsity incentive that makes CLTs compact also makes them capable of learning unfaithful circuits. These are not separable properties. You cannot have maximum compression and guaranteed faithfulness simultaneously under the current training objective. Choosing a lower sparsity coefficient helps but does not eliminate the problem. The community's response should be: treat attribution graphs as hypotheses that require ablation validation, not as ground truth descriptions of model computation. The current practice in many mechanistic interpretability papers is to present attribution graphs without ablation validation. That practice is now known to be insufficient.

Insight Two: The vision transformer application (arXiv:2604.13304) is a harder test than language models, and CLTs partially pass it, which is the most informative single benchmark in the current literature.

"Can Cross-Layer Transcoders Replace Vision Transformer Activations?" applies CLTs to ViTs (Vision Transformers) where the cross-layer structure is different from language models: patch tokens interact spatially, attention patterns are less dominated by positional structure, and MLP layers process spatially-distributed features. CLTs trained on ViTs achieve competitive replacement scores, meaning the cross-layer feature sharing principle is not specific to the sequential, causal structure of autoregressive language models. This matters because it provides evidence that CLTs are discovering something real about multi-layer residual networks in general, not just exploiting properties of text data or causal masking. The partial success and the documented failure modes in the ViT setting are both informative: CLTs capture some of the cross-layer structure but miss spatial feature interactions that are unique to vision models.

Transcoders provably outperform Sparse Autoencoders for mechanistic interpretability, and the community's continued focus on SAEs as the primary interpretability tool is not scientifically justified.

Paulo, Shabalin, and Belrose (arXiv:2501.18823, EleutherAI) evaluated transcoders and skip transcoders against SAEs across diverse models up to 2B parameters using SAEBench. Skip transcoders Pareto-dominate SAEs: they achieve lower reconstruction error AND higher automated interpretability scores simultaneously at all tested sizes (32,768, 65,536, and 131,072 latents). The improvement is not marginal. The paper explicitly concludes that "interpretability researchers should shift their focus away from sparse autoencoders trained on the outputs of MLPs and toward skip transcoders."

The reason transcoders outperform SAEs is structural. SAEs decompose intermediate activations, which are a mixture of past MLP outputs and unprocessed inputs to the current MLP. Transcoders model the functional behavior of the MLP itself, which is what researchers actually care about when asking "what is this computation doing." Despite this, SAEs dominate the interpretability literature because they are easier to train, have a longer research history, and the transcoder faithfulness concerns create legitimate uncertainty about when to trust their circuits. The correct response is not to return to SAEs. It is to use skip transcoders while developing better faithfulness diagnostics for CLTs.

CLTs are the best available tool for mechanistic interpretability of residual stream models at scale. The circuits they produce are compact, partially validated, and generalize across model types (language models, protein models, vision transformers). They enabled the most ambitious mechanistic interpretability project in the field's history. They are also capable of producing circuits that look correct while hiding the actual computation.

The correct response is not to abandon CLTs. It is to stop treating their output as ground truth. Every attribution graph is a claim about how a model computes, not a verified description of it. The claim needs to be tested by ablating the base model components the graph implies are relevant, and checking whether the graph predicted those ablations would matter. The mechanistic interpretability community has the tools to do this. The field's credibility depends on making it standard practice.