After working on computational pathology and anti-virulence discovery, I have converged on a short checklist for whether I believe my own results.
1. The split reflects deployment
Random splits are optimistic by construction. Family-held-out for proteins, external cohorts for clinical data — the split must simulate the hardest realistic use.
2. Calibration is reported, not assumed
Accuracy without calibration is a liability in biomedicine. Reliability diagrams and ECE cost little and reveal a lot.
3. Leakage has been actively hunted
Dedup by identity is not enough. Near-duplicates, shared scaffolds, batch effects — each needs a specific audit.
4. Claims are bounded in writing
Every strong sentence in the manuscript should map to a specific experiment, and every experiment should have a stated failure mode. If a claim cannot survive a structured null, it should not be made.
5. Someone could reproduce the disappointment
Reproducibility includes the negative results. A pipeline that can only reproduce successes is not reproducible.
This list is short on purpose. Each item is cheap compared to the cost of being wrong.
做过计算病理和抗毒力候选物筛选后，我逐渐整理出一份简短的检查清单，用来判断自己是否应该相信一个结果。
1. 数据划分要接近真实使用场景
随机划分天然容易得到乐观结果。蛋白质任务应考虑家族留出，临床数据应使用外部队列。数据划分要尽量模拟现实中最困难的使用情况。
2. 必须报告校准，而不是默认模型可靠
在生物医学任务中，只有准确率而没有校准是不够的。可靠性图和预期校准误差（ECE）的计算成本不高，却能暴露很多问题。
3. 要主动寻找数据泄漏
仅按完全相同的样本去重远远不够。近重复样本、共享分子骨架和批次效应都需要单独检查。
4. 写作时要限制论断强度
稿件中的每个强论断都应对应具体实验，每个实验也应说明可能的失效模式。如果一个论断无法通过结构化零假设检验，就不应该写出来。
5. 失败结果也应当可以复现
可复现性也包括阴性结果。只能复现成功案例的流程，并不是真正可复现的流程。
这份清单故意保持简短，因为完成这些检查的成本，通常远低于错误结论带来的代价。