Interpretability

Reverse-engineering what trained networks compute, from single features to whole circuits.

Open problems, most leverage first

All open problems →

#VotesProblemWork mixDVLStatus

010Seed-stable sparse autoencoder dictionaries: 80% shared features at 131K latentsReproductionSparse autoencoders for interpretability1 link2 bottlenecks
552639OpenD 55V 26L 39▲ 0
020SAE-based probes that match dense linear probes out of distributionReproductionSparse autoencoders for interpretability1 link2 bottlenecks
453533OpenD 45V 35L 33▲ 0
030Beating the best circuit-localization method on the MIB benchmarkReproductionMechanistic circuits and causal interventions2 bottlenecks
322419OpenD 32V 24L 19▲ 0

Research

Filter in Research →

Interpretability</> 2 reposTheoryActive

Mechanistic circuits and causal interventions

Methods that locate the subgraph of a network's components responsible for a behaviour, using activation patching and attribution, and check the result by intervention.

1 problem0 dependents0 maintainers

Interpretability</> 4 reposAlgorithmActive

Sparse autoencoders for interpretability

Dictionary-learning models trained on a network's activations to split them into sparse, often human-readable features; a main tool for finding units of computation in language models, with disputed practical value.