Chen Wenjie · Writing

How Should We Design a Structured Null?

Thu, 02 Jul 2026 00:00:00 GMT

Most “controls” in representation probing are too easy to beat. A random label shuffle destroys everything — including the structure your probe might legitimately exploit — so surviving it proves almost nothing.

The problem with naive shuffles

When we claim a model “encodes” a residue-level property, the implicit comparison is against a world where the property cannot be recovered from sequence. But sequences in the same family share statistical texture. A probe can succeed by exploiting family membership rather than residue signal.

A working definition

A structured null should:

Preserve global sequence statistics (length, composition, family structure).
Preserve the label’s marginal distribution.
Destroy the specific sequence→label coupling you claim the model captures.

def structured_null(seq_labels, family_ids, rng):
    """Shuffle labels within family, preserving family-level base rates."""
    out = {}
    for fam in set(family_ids):
        idx = [i for i, f in enumerate(family_ids) if f == fam]
        perm = rng.permutation(idx)
        for a, b in zip(idx, perm):
            out[a] = seq_labels[b]
    return out

Open questions

How adversarial should the null be? (Motif-preserving shuffles?)
Should the null be matched to each annotation type separately?

This is an ongoing note; I will update it as the SAE-DISTILL null battery stabilizes.

Interpreting Protein Language Model Features

Thu, 18 Jun 2026 00:00:00 GMT

Raw neurons in protein language models are polysemantic: a single dimension responds to hydrophobicity, secondary structure, and family artifacts at once. Sparse autoencoders (SAEs) promise monosemantic features — but the promise needs auditing.

Why SAEs over raw dimensions

Superposition: PLMs pack more concepts than dimensions. SAEs expand the basis.
Comparability: features are easier to align across layers and checkpoints.
Probe targets: a sparse feature is a cleaner unit for recoverability tests.

A minimal training sketch

loss = mse(x, decode(features)) + l1 * features.abs().mean()

The real work is not the loss — it is the evaluation of what the features mean.

What SAEs cannot tell you

Whether a feature is used downstream, or merely present.
Whether feature–label correlation survives family-held-out splits.
Whether the dictionary is stable across seeds.

Each of these needs its own experiment. Interpretability findings without structured evaluation are hypotheses, not conclusions.

What Makes a Biological AI Result Convincing?

Sat, 30 May 2026 00:00:00 GMT

After working on computational pathology and anti-virulence discovery, I have converged on a short checklist for whether I believe my own results.

1. The split reflects deployment

Random splits are optimistic by construction. Family-held-out for proteins, external cohorts for clinical data — the split must simulate the hardest realistic use.

2. Calibration is reported, not assumed

Accuracy without calibration is a liability in biomedicine. Reliability diagrams and ECE cost little and reveal a lot.

3. Leakage has been actively hunted

Dedup by identity is not enough. Near-duplicates, shared scaffolds, batch effects — each needs a specific audit.

4. Claims are bounded in writing

Every strong sentence in the manuscript should map to a specific experiment, and every experiment should have a stated failure mode. If a claim cannot survive a structured null, it should not be made.

5. Someone could reproduce the disappointment

Reproducibility includes the negative results. A pipeline that can only reproduce successes is not reproducible.

This list is short on purpose. Each item is cheap compared to the cost of being wrong.