陈文杰
Resume简历 Contact联系
Manuscript Submitted稿件已投稿 2026 Protein AI蛋白质 AI

SAE-DISTILLSAE-DISTILL

Can selected SAE-derived residue profiles be recovered from protein sequence beyond what is supported by matched structured-null controls?仅凭蛋白质序列，能否在匹配的结构化零假设对照之上恢复选定的 SAE 残基特征谱？

  • Protein Language Models蛋白质语言模型
  • Sparse Autoencoders稀疏自编码器
  • Interpretability可解释性
  • Evaluation评估
RESIDUE INDEX → 残基索引 → SAE FEATURES ↓ SAE 特征 ↓
Conceptual schematic · not presented as evidence概念示意图 · 不作为证据呈现

01Context研究背景

Why this project exists为什么开展这个项目

Testing whether sequence-only models can recover selected SAE-derived residue profiles from ESM-C representations.检验仅使用序列的轻量模型能否恢复 ESM-C 表征中选定的 SAE 残基特征谱。

02Question研究问题

The falsifiable question可证伪的问题

Can selected SAE-derived residue profiles be recovered from protein sequence beyond what is supported by matched structured-null controls?仅凭蛋白质序列，能否在匹配的结构化零假设对照之上恢复选定的 SAE 残基特征谱？

03Method研究方法

How the question is tested如何检验这个问题

  1. M01 SAE-derived residue profiles from ESM-C representations基于 ESM-C 表征构建 SAE 残基特征谱
  2. M02 Sequence-only lightweight model training训练仅使用序列的轻量模型
  3. M03 Matched structured-null evaluation匹配的结构化零假设评估
  4. M04 Held-out evaluation and evidence-bound interpretation留出评估与证据边界控制

The pipeline, step by step流程，一步一步看

Scroll through the six stages — or read them as a plain list. Nothing here traps your scroll.可以滚动着看这六个阶段，也可以当成普通列表读。这里不会劫持你的滚动。

  1. Step 1 / 6第 1 步 / 共 6 步

    Protein sequence蛋白质序列

    Raw amino-acid sequence is the only input — no structure, no annotations.唯一输入是原始氨基酸序列，不使用结构或注释。

    MKVLWAALLVTFLAGCQAKVEQAVETEPEPELRQQTEWQ…MKVLWAALLVTFLAGCQAKVEQAVETEPEPELRQQTEWQ…

  2. Step 2 / 6第 2 步 / 共 6 步

    Language model embedding语言模型嵌入

    Each residue is mapped to a high-dimensional vector by the PLM.PLM 把每个残基映射为一个高维向量。

    h_i ∈ R^d, one vector per position ih_i ∈ R^d，每个位置 i 对应一个向量

  3. Step 3 / 6第 3 步 / 共 6 步

    SAE feature activationSAE 特征激活

    A sparse autoencoder decomposes each embedding into interpretable features.稀疏自编码器把每个嵌入分解为可解释特征。

    f_i = TopK(W·h_i), sparse feature vectorf_i = TopK(W·h_i)，稀疏特征向量

  4. Step 4 / 6第 4 步 / 共 6 步

    Residue-level profile残基级轮廓

    Feature activations form a per-residue profile across the sequence.特征激活沿序列形成残基级轮廓。

    activation(position) curves per feature每个特征的 activation(position) 曲线

  5. Step 5 / 6第 5 步 / 共 6 步

    Structured null comparison结构化零假设对照

    The same pipeline runs on structure-preserving shuffles. Real signal must win.在保留结构特征的置乱数据上运行同一流程，真实信号必须胜过对照。

    real labels vs. family-preserving shuffles真实标签 vs. 保留家族结构的置乱标签

  6. Step 6 / 6第 6 步 / 共 6 步

    Held-out evaluation留出评估

    Recovery is re-tested on protein families never seen during training.在训练中从未见过的蛋白质家族上重新测试可恢复性。

    family-held-out splits — generalization test家族留出划分：泛化测试

04Evidence证据

What the current evidence supports现有证据支持什么

  • The lightweight sequence model recovered stable local sequence patterns in the selected profiles.轻量序列模型能够学习选定特征谱中的稳定局部序列规律。
  • Matched controls were used to test whether recovery exceeded signal attributable only to profile statistics.使用匹配对照检验恢复结果是否超出仅由特征谱统计分布所支持的水平。

05 Limitations局限性

Where the claim stops论断止于何处

  • The study evaluates selected SAE-derived profiles; it does not establish that all SAE features have biological meaning.研究只评估了选定的 SAE 特征谱，不能据此推断所有 SAE 特征都具有生物学含义。
  • Recoverability does not by itself show that a feature is causally used by the protein language model.可恢复性本身不能证明蛋白质语言模型会因果性地使用这些特征。

06 Next steps下一步计划

What comes next接下来做什么

  • Respond to peer review and document the evaluation workflow for reproducibility.根据同行评审意见修订，并进一步整理可复现的评估流程。