The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-2429
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-08T14:01:01.159547Z
status: published
lens: g_space
sector: science
headline: Sparse autoencoders uncover biologically interpretable features in protein language model representations
dek: Foundation models in biology-particularly protein language models (PLMs)-have enabled ground-breaking predictions in protein structure, function, and beyond. However, the "black-box" nature of these representations limits transparency and explainability, posing challenges for human-AI collaboration and leaving open questions about their human-interpretable features. Here, we leverage sparse autoencoders (SAEs) and a variant, transcoders, from natural language processing to extract, in a completely unsupervised f…
gain_title: In a PNAS study of the ESM2 protein model, sparse autoencoder features reached median interpretability scores of about 0.7, versus just above 0.5 at best for ESM2's own neurons.
problem_title: (none)
trace_subject: (none)
gain_reading: In a PNAS study of the ESM2 protein model, sparse autoencoder features reached median interpretability scores of about 0.7, versus just above 0.5 at best for ESM2's own neurons.
gain_evidence: ESM neurons exhibit median Pearson correlations slightly above 0.5 at best, whereas sparse features consistently achieve median correlations around 0.7
problem_reading: (none)
problem_evidence: (none)
quick_read: Researchers trained sparse autoencoders and a variant called transcoders on internal representations from layers 6 to 10 of ESM2, a protein language model, to pull out features a person can read. Across layers, the autoencoder features had median interpretability scores near 0.7, against just above 0.5 at best for ESM2's own neurons.

Many features lined up with specific biology. Examples include the NAD kinase family, the IUNH family, the PTH family, methyltransferase activity, and olfactory and gustatory sensory perception. In a Gene Ontology analysis of 18,142 human proteins, features were tied to terms at every level of the GO hierarchy and sat closer together in the GO graph than random terms.

The authors say the approach could help researchers explain and debug protein predictions and audit open-source models for dangerous biological information. They note these unsupervised methods are not built to predict protein function, and only 200 of 20,000 features per model were interpreted.
limitation: Due to API constraints, only 200 of 20,000 sparse features were interpreted per model, scoring used an LLM-based proxy, and consistency across random initializations was not assessed.
tag: Evidence-backed gain
key_points: Sparse autoencoders had a hidden layer of 20,000 features and were trained on 480-dimensional representations from layers 6 to 10 of the 35-million-parameter ESM2 model. | Training drew on 50 million UniRef50 protein sequences, split 80-20 into training and validation sets. | For protein-level features, median interpretability scores were near or above 0.7 across layers; for amino acid-level features they were 0.673 to 0.714. | ESM2 neurons peaked just below 0.5 at amino acid level and just above 0.5 at protein level. | Transcoders scored 0.66 to 0.73 at protein level, comparable to the autoencoders. | For layer 9, 52.17% of biological-process GO terms in the analysis set were enriched by at least one feature; layer 9's autoencoder and transcoder shared 9,180 enriched GO terms. | For layer 9's amino acid-level autoencoder, 50.50% of sampled features cleared a 0.7 interpretability threshold and 37.50% cleared 0.8. | Coactivating proteins had higher sequence similarity than non-coactivating ones, but similarity correlated only weakly (0.117 to 0.263) with interpretability.
rundown: Interpretations were automated with Anthropic's Claude 3.5 Sonnet. For each feature, Claude wrote a one-sentence description from 8 active and 7 inactive proteins, then predicted activations on 9 active and 8 inactive held-out proteins. The correlation between predicted and actual activations was the interpretability score. The same protocol was applied to 200 sampled ESM2 neurons for comparison.

Individual features mapped to specific biology: feature 2579 in layer 7 to RNA and DNA methyltransferases, feature 15683 in layer 10 to Complex I of the electron transport chain, feature 12680 in layer 9 to caspases in apoptosis, and feature 19717 in layer 9 to the uncharacterized UPF0312 family.

The authors list several limits. Only mean-pooled protein-level representations were tested, scoring relies on an LLM proxy sensitive to prompts and metadata, and feature stability across random seeds was not assessed. API constraints limited interpretation to 200 of 20,000 features per model. They also say these methods are not meant to predict protein function.
sources:
- peer_reviewed | Proceedings of the National Academy of Sciences | https://doi.org/10.1073/pnas.2506316122 | 2025-08-19
prev: 0000000000000000000000000000000000000000000000000000000000000000

SHA-256 hash

bc0d6878d17846367e66fdcec4708fa6c279b0f4a008f5c8caa7abba1ddcc49f

Previous version’s hash

0000000000000000000000000000000000000000000000000000000000000000