Sampling scales. Selection does not.

How this was measured

What this is

Four independently trained predictors, each sampled to 512 structures per target on the 2026ARK-AB benchmark, every structure scored with DockQ against the experimental reference. Boltz-2, OpenDDE-abag and Protenix-v2 are AF3-style all-atom diffusion co-folders; ESMFold2 is a single-sequence folder. – labelled samples, folded on Tenstorrent Wormhole Galaxy hardware.

They do not fail on the same targets: the pairwise overlap of their failure sets is – (Jaccard), and – are failed by all four. That partial independence is what the compute-split section buys.

Prior work

That deep sampling raises the ceiling faster than confidence ranking delivers is already published for antibody–antigen complexes, not new here. Fromm et al. (Bioinformatics 2026) report best-of-N mean DockQ rising from below 0.3 to above 0.5 by 200 samples across 110 targets, and name identifying the best generated model as the remaining bottleneck. OpenDDE's technical report measures the same ranked-versus-oracle gap at its default N = 5.

This panel adds that gap expressed as an effective N, the split of ranking failure into within-target and across-target, the finding that most failures are epitope-level rather than pose-level, the price of model diversity against sampling depth at measured compute, and a pre-declared null on fixing selection with the scores a model already returns.

Limitations, denominators and provenance

• The panel is complete: – model–target cells, every one – samples deep by construction, so the headline N is not an average over a ragged panel. Two exclusions, neither of them missing data. Three of the 164 targets (9ly2, 9ly3, 9lz2) are anti-phosphoepitope antibodies: every contact atom on the antigen side of their interface sits on a phosphoserine, and DockQ scores only standard residues, so the interface has nothing to score and they carry no labels in any model. One cell is dropped as a mis-fold: OpenDDE-abag on 9sbb lands in a pTM 0.67–0.70 basin against 0.91 for the same input refolded, the only such case in a scan of 41 paired targets. Per model: –
• Code provenance. The campaign ran one frozen engine tree, so every cell is comparable. – cells, on the four largest targets, needed tensor-lifetime fixes absent from that tree and ran on later trees, each cell entirely on one. Every headline is computed with those cells in and out: the largest movement is –, and –.
• The oracle is an upper bound, not an achievable number, and it cannot leak into the delivered number: the two curves differ only in what they sort by, and the one tie-break that touches both resolves against the selector.
• Pooling checks. Truncating each 512-sample pool to its first 256 reproduces the earlier 256-sample round exactly on –; the rest have no complete 256 rung to nest against. Repeated structures within a pool: –. Seeds are per model and per chunk, so independence comes from the inputs differing rather than the draws: the noise tensor is target-shaped, and the four models' seed blocks are disjoint.
• Epitope and CDR claims are quoted at their own depth, never at 512. Those labels are null on a handful of named targets where the quantity does not exist, so they are read at their own depth. Mean samples labelled per target: –
• One selector is stored on a coarse grid: –, so ties are common, and our tie-break credits it with the worse of two samples it cannot separate. That is the direction that flatters our headline, so we measured it: resolving ties the other way moves its delivered DockQ by –, far inside the interval.
• Cost basis. – analysed samples carry no measured wall time, so the cost model takes a per-target median.
• Four specific models at specific settings. ESMFold2 runs single-sequence; OpenDDE-abag is antibody-specialised, so its lead is not a general co-folding result. This measures each model's own shipped confidence, not published rankers: the PAE-based scores (pDockQ2, ipSAE, AntiConf) cannot be computed here because PAE matrices were not written during the campaign, and learned rankers (DeepRank-Ab, ABAG-Rank) were not run, so whether one closes the gap is untested.

Reproduce

• Models: Boltz-2, OpenDDE-abag, Protenix-v2, ESMFold2 as shipped in TT-Bio at commit 6d352084c, stock checkpoints, inference only.
• Targets: the 2026ARK-AB benchmark, 164 antibody–antigen complexes, 161 of them scorable, each with an experimentally solved reference.
• Depth and seeds: 512 samples per target per model, run as eight 64-sample chunks. Base seed 40000 Boltz-2, 20000 OpenDDE-abag, 30000 Protenix-v2, 50000 ESMFold2; chunk j uses base + 1000·j, so the shallower rungs are exact prefixes of the deeper ones: the 512-sample pool is a strict superset of every rung below it.
• Settings: the three co-folders fold with MSAs served from a precomputed cache (--msa_cache_only --max_parallel_samples 5, Boltz-2 8); ESMFold2 is single-sequence (--single_sequence --recycling_steps 10 --sampling_steps 100); --host_threads 8 throughout. Everything else as shipped by each model.
• Hardware and cost: one 32-chip Tenstorrent Wormhole Galaxy. Card-hours are measured, not modelled: – per sample.
• Metrics: DockQ against the reference, 0 to 1. Epitope overlap is the Jaccard index of predicted against true antigen contact residues. Effective N is where the best-of-k curve crosses the delivered level.
• Run it: python3 analysis/build_insights.py -o data/insights.json rebuilds every number on this page from the per-sample table, one row per structure with its DockQ, confidence and epitope labels. That table is published in full at Tenstorrent/abag-xm; the aggregates this page reads are committed in data/insights.json.

Full statistics, every interval and every control are in the findings document.

What this is built on

The result belongs as much to the people who released these models and tools. Every prediction here was made by someone else's model, scored by someone else's metric, against structures someone else solved and deposited.

• The target set is 2026ARK-AB, the antibody-antigen benchmark released with OpenDDE: 164 PDB targets, 404 interfaces, 159 clusters at 40% MMseqs2 entity clustering. We did not assemble it. See Folding, Reasoning, and Scaling with Open-source Drug Discovery Engine and the target list in aurekaresearch/OpenDDE under benchmarks/2026ARK_AB/. AbAg-XM is our name for the predictions and labels we generated on top of it, not for the benchmark.
• Boltz-2, jwohlwend/boltz, MIT.
• Protenix-v2, bytedance/Protenix, Apache-2.0.
• OpenDDE, aurekaresearch/OpenDDE, Apache-2.0.
• ESMFold2, biohub/ESMFold2, MIT, subject to the Biohub Acceptable Use Policy.
• DockQ, the scoring metric used throughout, bjornwallner/DockQ.
• Reference structures from the Protein Data Bank, CC0. Please cite the original depositors of any structure you use.
• Alignments built from UniRef30 via ColabFold.

All four models were run as shipped, on stock checkpoints, with each model's own default settings. No model was tuned, retrained or recalibrated for this benchmark.

@misc{abagxm2026,
  title  = {AbAg-XM: 335,360 DockQ-labelled antibody-antigen structure predictions
            from four models},
  author = {Th\"uning, Moritz},
  year   = {2026},
  url    = {https://huggingface.co/datasets/Tenstorrent/abag-xm},
  note   = {Analysis: https://moritztng.github.io/abag-scaling/}
}