Intermediate results obtained

The current best: jepa-x12

jepa-x12 is the student of a twelve-model ensemble (see distillation). It is the model frozen for Review 2 and the app's default 260-sign model from 27 September (it replaced jepa-rr4).

The accuracy test

The accuracy test (called "the gate" in the earlier reports) is the clip-weighted top-1 accuracy over 599 clips: RKMVU-220, one signer who appears nowhere in training, and CISLR test-role 379, clips the model is never trained on. One clip is 0.17 pp, so differences under about half a point are single clips. Because CISLR's test clips share presenters with its training clips, the RKMVU number is reported beside every accuracy-test number — it is the only one that measures a true stranger.

Model lineage

All numbers fp32 unless marked; top-1.

Where the gain came from

The accuracy test rose by +9.2 pp (0.4591 → 0.5509). Two causes can be separated because each was measured on its own:

1. The dataset contract: +7.0 pp over four seeds. The same recipe was trained four times on the old pack and four times on the reviewed v4 pack, in the same run:

   Pack	Seed 1	Seed 2	Seed 3	Seed 4	Mean
   Control (champion recipe, old pack)	.4057	.4174	.4124	.4107	.4116
   Reviewed v4 pack	.4825	.4942	.4691	.4808	.4817

   All four seeds went up. The review was about sense conflation — whether the clips filed under one word really show one sign, and whether two words are really the same sign — see the dataset review loop.

2. Ensembling and distillation: the rest. Four-seed ensembles of the same two packs:

   Ensemble	Accuracy test	RKMVU	CISLR
   Control, 4 seeds	.4574	.4727	.4485
   v4 pack, Phase B, 4 seeds	.5376	.4909	.5646

   Twelve fine-tunes distilled into one student (jepa-x12) took it to .5509.

What the app shows

The harness scores fp32 models; the app runs the int8 model with the mirror pass. Scored the way the app runs them, on the same 599 clips:

In Practice mode a sign is credited when its calibrated confidence clears 0.5. At that floor jepa-x12 credits 69% of correct signs at 83% precision; jepa-rr4 credited 61% at 82%. More correct attempts are recognised, and a credited attempt is slightly more likely to be right.

The honest reading

• The Review 1 target (≥ 0.50) is met on the accuracy test (0.5509) and borderline on the unseen signer (0.5000 fp32, 0.4909 int8).
• The ensemble is still ahead of its student (0.5576 vs 0.5509), so a better student is possible without new data — but on the unseen signer the gap is 0.5045 vs 0.5000, about one clip.
• Every claim on this page rests on at least four seeds.

Negative results kept

Each of these was a reasonable idea, measured properly, that did not help. They are kept because each one removes a direction from the plan.

Shape match, measured as a helper

Shape match has no learned weights, so it was tested in the two roles where it could help the AI model rather than compete with it:

A shortlist helps shape match by almost ten points, but it never overtakes the model it is reading from, and as a label checker it is worse than the model's confidence. It stays in the product as a transparent second opinion and as a control, not as a training signal.

What the data review found

• ISLRTC dictionary: 64% of files are longer than 8 s — lessons, not single words; 17% are silently cut at 36 s; 767 "words" exist only as file names.
• INCLUDE: 76% of frames are idle (before or after the sign).
• Vision review: 75 escalations; 35 settled by blind review of the real videos (21 by INCLUDE's own ground truth, 6 confirmed different signs, 8 rejected as the same sign); 40 open in the Review tab for a human decision.