EgoHand4D · Technical Report
In-the-Wild Inference
6. In-the-Wild Inference
The promoted lineage on real wild video (blue = left, red = right; presence + visibility gated with inference-side latching; clips below rendered at the uf12 generation — the gallery runs the full lineage with the champion v11 as the default view, and v10 kept viewable under its honest wild-regression label). No wild 3D GT — these are qualitative evidence; the quantitative gates are the GT + wild-stability instruments in §4.3.
Data-integrity fix (2026-07-01): earlier "raw" clips formedical/shoeswere actually skeleton-overlaid output videos (inference had run on already-annotated frames, contaminating the benchmark). All raw sources are now confirmed skeleton-free and expanded to 8 clips covering ego first-person + fine fingers + bimanual collaboration + fast motion / off-frame. Full 8 clips × lineage is in the model iteration gallery.
| Clip | Scene | present (uf12) | Behavior |
|---|---|---|---|
ego_wire | wiring assembly · first-person · gloved industrial | L 100% / R 87% | stable ego two-hand; foreshortened right still splays slightly (known frontier) |
ego_desk | desktop reach · ego overhead · bare hands | L 100% / R 97% | stable overhead two-hand, good fast-motion follow |
guitar | guitar · fine finger joints | L 100% / R 21% | fretting hand stable; fast strumming hand occasionally lost |
coffee | pour-over (French press) · bimanual | L 65% / R 100% | good collaboration tracking, plunger hand steady |
drums | drumming · fast motion + stick occlusion | L 100% / R 50% | holds the main hand under fast strikes + stick occlusion |
conductor | conducting · arms wide · large off-frame | L 40% / R 29% | OOD large off-frame; present correctly drops as hands leave |
Persistence upgrade (post-table): the present percentages above predate the §1.6 presence-recall upgrade (hysteresis + sustained mid-confidence latch + joint-based off-frame test). With it, the previously-lost hands recover — both-hand recall guitar-R 0% → 100%, medical 100%, ego_wire 98% — while the OOD true negatives (conductor / speaker) stay correctly gated: no fabricated skeletons.
Full-pipeline joint visualization (raw → 2D → 3D, 4 panels synchronized) — the egocentric main line:
Takeaway — Strong on ego first-person two-hand, bimanual collaboration, and no-hand specificity; the honest frontiers are fast strumming/striking, foreshortened reach, and large off-frame recall — exactly what §2 occlusion data and §7 re-distillation target.