How Should We Design a Structured Null?

Most “controls” in representation probing are too easy to beat. A random label shuffle destroys everything — including the structure your probe might legitimately exploit — so surviving it proves almost nothing.

The problem with naive shuffles

When we claim a model “encodes” a residue-level property, the implicit comparison is against a world where the property cannot be recovered from sequence. But sequences in the same family share statistical texture. A probe can succeed by exploiting family membership rather than residue signal.

A working definition

A structured null should:

Preserve global sequence statistics (length, composition, family structure).
Preserve the label’s marginal distribution.
Destroy the specific sequence→label coupling you claim the model captures.

def structured_null(seq_labels, family_ids, rng):
    """Shuffle labels within family, preserving family-level base rates."""
    out = {}
    for fam in set(family_ids):
        idx = [i for i, f in enumerate(family_ids) if f == fam]
        perm = rng.permutation(idx)
        for a, b in zip(idx, perm):
            out[a] = seq_labels[b]
    return out

Open questions

How adversarial should the null be? (Motif-preserving shuffles?)
Should the null be matched to each annotation type separately?

This is an ongoing note; I will update it as the SAE-DISTILL null battery stabilizes.

很多表征探针使用的“对照”太容易被击败。随机打乱标签会破坏所有信息，包括探针本来可以合理利用的结构。因此，优于这种对照并不能证明太多。

简单置乱的问题

当我们说模型“编码”了某种残基级属性时，隐含的比较对象应当是一个无法从序列恢复该属性的世界。但同一家族中的序列会共享统计特征。探针可能只是识别了蛋白质家族，而不是真正找到了残基级信号。

一个可用的定义

结构化零假设应满足以下条件：

保留整体序列统计特征，例如长度、组成和家族结构。
保留标签的边际分布。
破坏我们声称模型捕获到的特定“序列 → 标签”关系。

def structured_null(seq_labels, family_ids, rng):
    """在家族内置乱标签，同时保留家族层面的基础比例。"""
    out = {}
    for fam in set(family_ids):
        idx = [i for i, f in enumerate(family_ids) if f == fam]
        perm = rng.permutation(idx)
        for a, b in zip(idx, perm):
            out[a] = seq_labels[b]
    return out

尚未解决的问题

零假设需要有多强的对抗性？是否要保留基序？
不同注释类型是否需要分别设计零假设？

这是一篇持续更新的研究笔记。我会随着 SAE-DISTILL 的零假设测试逐步稳定而继续修订。

How Should We Design a Structured Null?如何设计结构化零假设？