aceSparse autoencoders uncover biologically interpretable features in protein language model representations
Foundation models in biology-particularly protein language models (PLMs)-have enabled ground-breaking predictions in protein structure, function, and beyond. However, the "black-box" nature of these representations limits transparency and explainability, posing challenges for human-AI collaboration and leaving open questions about their human-interpretable features. Here, we leverage sparse autoencoders (SAEs) and a variant, transcoders, from natural language processing to extract, in a completely unsupervised f…
In a PNAS study of the ESM2 protein model, sparse autoencoder features reached median interpretability scores of about 0.7, versus just above 0.5 at best for ESM2's own neurons.
Due to API constraints, only 200 of 20,000 sparse features were interpreted per model, scoring used an LLM-based proxy, and consistency across random initializations was not assessed.
Sources
- Peer-reviewedProceedings of the National Academy of Sciences2025-08-19
What do you make of this one?
Truvace Impact Record TRV-2026-2429, v1: “Sparse autoencoders uncover biologically interpretable features in protein language model representations.” Truvace, 2026-11-08. /record/TRV-2026-2429 (accessed on the day you copy it). sha256 bc0d6878d1784636…
Change history
Every change made to this record since it was first published, and why. Nothing has changed since it went up.
Certified into the record
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-2429 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.