Comparison vs Dyn-HaMR

5. Comparison vs Dyn-HaMR

Dyn-HaMR is a recent optimization-based method for dynamic multi-hand 4D recovery — per-frame HaMeR plus a per-sequence root/smoothness optimization and a SLAM-estimated camera. Implementation provenance: all Dyn-HaMR numbers in this section come from the somantis production-hardened fork (depth caps, non-finite rollback, production loss weights, ViPE camera backend — a strict superset of the upstream release's stability), run in two intrinsics regimes: its own estimated K and pinned exact GT K (§5.4). We compare it to our feed-forward pipeline on eight egocentric two-hand clips. To keep the comparison honest we do three things: draw both methods through one shared renderer; score 2D against a dual-agreement referee (MediaPipe hands confirmed by an MP-box-seeded raw-HaMeR 2D — MediaPipe alone is NOT independent of us, since MediaPipe pseudo-labels are in our training mix); and score 3D against exact GT MANO on held-out egocentric lab sequences (§5.3), where no referee is needed at all.

**CORRECTION (2026-07-04) — a bug in our harness was crippling the baseline; every Dyn-HaMR
number in this section has been re-run.** Visual QA of the side-by-sides flagged Dyn-HaMR's
meshes as implausibly bloated. Root-causing it against Dyn-HaMR's own overlays found the fault
on our side of the fence: ViPE (the fork's camera backend) processes a height-480 downscale
of the video and emits intrinsics at that resolution, and the camera loader only rescaled them
back to full resolution when an explicit resolution field was present — which ViPE never
writes. Every prior run therefore optimized full-resolution keypoints with a proc-resolution
focal — 4.17× too small on 2800×2000 ARCTIC, 1.5× on 720p H2O, 1.12× on 540p wild — which
mechanically drags hands toward the camera and blows up betas (the "melted mesh" / 0-converged
pathology previously attributed to the method). After fixing the loader we re-ran every
Dyn-HaMR artifact (both intrinsics regimes on the GT windows, all 8 wild clips) and re-derived
every number, table, still and video below. The corrected baseline is substantially stronger
(it now converges 6/6 with pinned GT K, and its absolute depth beats ours there); our lead on
articulation, root-relative accuracy, reprojection, temporal stability and speed stands, and is
now fairly established. Pre-fix artifacts are archived under *_kbug/ for audit.

5.1 One renderer, each method's own camera

Every comparison feeds a single renderer (hand2d/viz_panels.py) and lands in one task-aligned composite per clip (hand2d/bench_compare.py) — each method is compared on the task it actually performs: row 1 (2D tracking) = OURS Stage-1 2D skeleton vs MediaPipe 2D, drawn in the identical per-finger style; row 2 (3D) = OURS MANO-AR overlay + 3D orbit + ego-camera frustum; row 3 (3D) = Dyn-HaMR, its world_results.npz transformed into the camera frame and projected with its own intrinsics through the same renderer, so its AR panel reproduces its native overlay. The only thing that differs on screen is each method's own geometry. Read the 3D panels separately: the AR overlay is 2D placement; the 3D orbit is the true metric geometry. Every comparison output is additionally stamped with each method's temporal mode (ours = bidirectional_window+temporal_opt, Dyn-HaMR = batch_optimization, MediaPipe/raw HaMeR = frame_independent), so smoothness-derived metrics are never read without that context. The full 23-clip grid lives in the gallery; four representative clips below.

5.2 Results (8 clips, fairness-corrected referees)

Referee honesty. MediaPipe pseudo-labels are part of OUR training mix (WildPseudoDataset), so MediaPipe is not an independent referee for us — it is (partially) our teacher. The headline 2D row is therefore scored against a dual-agreement referee: MediaPipe hands confirmed by an MP-box-seeded raw HaMeR 2D (independent of our Stage-1; _bench/hamer_ref.py); frames where the two estimators disagree are dropped. Presence is scored by referee-hand recall (how often a method's track lands on a referee hand) instead of raw both-hand rate, which structurally favors Dyn-HaMR (it always emits 2 tracks). The full per-clip table (now 8 egocentric clips: ego_desk, ego_wire, electronics_soldering, jewelry_brooch, lab_cells, medical_tubing, pottery, textile_sewing) is below; inputs are hash-frozen in _bench/protocol_v1.json (freeze_protocol.py --verify).

Median across the 8 clips (ours / Dyn-HaMR, corrected-K reruns on both sides):

Ours leads on 2D accuracy against the referee both methods are independent of (6/7 scoreable clips; corrected Dyn takes lab_cells 35.6 vs 55.8 px — one of the two clips where it converges), temporal stability (~5× less articulation jitter), and speed (measured end-to-end ≈3–19× per clip). The MediaPipe row is now a statistical tie (45.9 vs 45.1 px) — kept only for continuity, since MP is our own teacher and the dual referee is the honest 2D instrument. The corrected Dyn also takes unmatched-prediction rate (0.06 vs 0.12 — it always emits exactly two tracks). On scale the honest claim on wild footage is stability, not accuracy: our clip-constant betas hold a consistent 11–17 cm hand across every wild clip, but there is no wild GT to certify it — against exact lab GT (§5.3) our hand length is accurate to +0.9 cm median, vs Dyn's −6.3 cm deficit. In the wild — even after the K-bug correction — Dyn-HaMR's optimization remains the failure mode on most clips: with the corrected loader and its own estimated intrinsics it converges on only 2 of 8 clips (lab_cells |β|max 1.1, pottery 1.3 — genuinely clean fits, reported as such), leaves the betas prior on five (|β|max 4.8–16.1), solves a negative focal length on medical_tubing (camera divergence), and still produces no tracks at all on electronics_soldering. The contrast with §5.4 is the point: hand the same fork perfect lab intrinsics and it converges 6/6 — its wild failure is its dependence on a self-estimated metric camera, which is precisely what egocentric wild footage never provides. (The fork also ships per-frame failure masking and depth-bias auditing as product features — its own maintainers treat per-frame failure as an expected operating mode in the wild.)

Dyn-HaMR is competitive on raw 2D placement where its camera holds and its optimization converges — on lab_cells (one of its 2 converged clips) it beats us outright on the dual referee, 35.6 vs 55.8 px — and on track persistence: it always emits both tracks, whereas ours presence-gates and dropped a hand on jewelry_brooch (both-rate 0.71 there).

Referee coverage caveat. The dual-agreement referee only exists where MediaPipe and raw HaMeR agree — 0–94% of hands depending on clip difficulty (0% on electronics_soldering, where MediaPipe finds 21 hands in the whole clip). Hard-frame performance is therefore NOT measured by this table at all; it is measured by the GT evals below, which is why they — not this table — are the gate.

5.3 GT ego head-to-head (exact GT MANO, real K — no referee)

The referee-based table above cannot see the hard frames (the referee vanishes exactly there), so the gate is the GT-3D instrument from §4.3 (hand2d/eval_gt3d.py): 27 held-out egocentric windows (one per ARCTIC-VAL / H2O-VAL sequence, 120 frames each, frozen in local_2d/gt3d_eval/set_v1.json) scored directly against exact GT MANO joints with the real calibrated K. Ours (v11 champion) over all 27 clips (6,480 GT hand-frames, median/p90): hand recall 1.000, MPJPE (absolute, carries monocular depth error) 76/125 mm, root-aligned 18.2/26.5 mm, Procrustes-aligned 8.9/13.8 mm, reprojection-to-GT 5.8% of hand-bbox diagonal, root-relative acceleration error 1.1 mm, hand-length error vs GT +0.9 cm, both-hand recall 0.997, swap rate 0.0000.

Dyn-HaMR on the identical GT clips (hardened fork, corrected-K rerun, estimated-K regime; 6-window subset, same mp4s, pooled medians — ours / Dyn): root-aligned MPJPE 16.3 / 34.9 mm, PA-MPJPE (Sim(3), with scale) 8.0 / 27.3 mm, reprojection 0.044 / 0.167 of hand diag, acceleration error 1.2 / 3.1 mm, hand-length error +0.3 / −6.5 cm. With exact GT and no referee, ours wins every quality axis by 2–4×; recall/swap are a wash (both ≈ perfect on these easy-presence lab clips). Convergence after the K fix is no longer the story on lab footage: the corrected fork converges 5/6 (est-K) and 6/6 (pinned GT K) with healthy betas (|β|max ≈ 0.9–1.8) — the earlier "0-converged" pathology was the harness K bug, not the method (see the correction note above). What remains structural for Dyn on this footage: the

3.4× articulation gap, a −6.3 cm measured hand-length deficit (the weak-perspective scale-for-depth residual), and — in the est-K regime — absolute depth failures up to 1.4 m (ViPE cannot estimate a 2414-px tele focal from a 480p downscale; est-K abs-MPJPE median 187 mm vs 43 mm pinned).

Enriched rescore — the new instrument axes. The GT-3D instrument now reports camera-space global error (G-MPJPE / gravity-aligned GA-MPJPE), both Procrustes conventions labeled explicitly (Sim(3) with scale and SE(3) without — SE(3) costs us ≈ +1.7 mm, 10.5 vs 8.8 median, and makes our PA numbers directly comparable to SE(3)-PA numbers from other codebases), jerk error, relative-rotation error, a per-finger breakdown, tracking continuity, and per-clip depth bias. Support caveat, stated plainly: the Dyn columns are its 6-window run; ours is the full 27-window set (the enriched ours run scored v10, whose GT-3D is at parity with the champion v7 — §4.3). A same-support 3-way version of this table — ours re-pooled on the identical 6 windows, Dyn est-K, Dyn pinned-GT-K — is in §5.4. Highlights (overall median, ours / Dyn):

Continuity is the one axis Dyn takes: it always emits both tracks (the same structural property as its both-hand rate in §5.2), while our presence gating cost two frames across 6,480 — we report that honestly.

Depth bias — the corrected finding (and the one axis the corrected baseline wins). Comparing each method's mean hand depth in its own camera frame against GT: in the est-K regime the corrected Dyn still misses badly (median +151 mm, mean |Δ| 420 mm, blow-ups to +1.4 m — the focal it estimates from a 480p downscale cannot carry metric depth), but with pinned exact GT K its per-sequence optimization nails absolute depth: median +7 mm, mean |Δ| 29 mm. Ours carries +83 mm median (a mild far bias, mean |Δ| 71 mm) — better than est-K Dyn by 6×, but ~2.4× worse than pinned-K Dyn. Stated plainly: when the baseline is handed perfect intrinsics, its global optimization beats our feed-forward depth placement, and removing our +8 cm far bias is now the clearest single roadmap item this comparison produces (§7). Source: local_2d/gt3d_eval/depth_report.json.

Full numbers: local_2d/gt3d_eval/results_ours_v11.json (champion, 27 windows; results_ours_v7/ v10.json kept for lineage) + results_dyn.json (corrected est-K) + results_dyn_hardened.json (corrected pinned-GT-K, §5.4) + depth_report.json. Pre-fix result files are archived under dynhamr_*_kbug/. The complementary GT-2D frontier gate is defined in §4.3.

5.4 Strongest-baseline run — their fork, their guards, their config, perfect K, corrected harness

The last available defense of Dyn-HaMR on this footage was "you never gave it its best shot." So we did, definitively — and when our own visual QA showed its meshes looking implausibly bad, we audited our harness, found the proc-res-K bug (correction note, §5 top), fixed it, and re-ran both regimes: the somantis production-hardened fork — max-depth cap, production loss weights (joints2d = 20000, joints3d_smooth = 10000), non-finite rollback, ViPE camera backend — on the same 6 GT windows, with its own estimated K and with exact GT intrinsics pinned (verified: the K reaching the optimizer now equals GT K exactly). Same renderer, same scorer; for a clean same-support read, ours (v11 champion) is re-pooled over the identical 6 windows. Source: hand2d/_bench/out/dyn_hardened_table.tsv + local_2d/gt3d_eval/results_dyn_hardened.json.

What the corrected run changes — and what it doesn't. With the harness bug fixed the baseline is transformed on the global axes: it converges (6/6 pinned, 5/6 est-K, betas |β|max 0.9–1.8 — the old "0/6, betas to ±30" rows were our K bug, not the method), root-aligned error halves (59.6 → 34.0 mm), and with pinned GT K its absolute depth becomes excellent — mean |Δ| 29 mm, better than our 71 mm (our +8 cm far bias is now the clearest single item this comparison hands our roadmap). What the fix does not change is exactly the part that is structural to the method: articulation stays at PA ≈ 27 mm in every regime, bugged or fixed, est-K or perfect K (ours 8.0 mm — a 3.4× gap), the measured hand length stays ~6 cm short (the weak-perspective scale-for-depth residual of its HaMeR initialization), jerk stays ~3× ours, and runtime stays ~3 min per 120-frame clip vs our ~0.4 s/frame pipeline. Without GT K — the only regime that exists outside a lab — its estimated focal cannot carry metric depth (abs-MPJPE 187 mm, blow-ups to +1.4 m), and on wild footage (§5.2) its optimization still degenerates on 5 of 7 tracked clips. This is the answer to "did you give Dyn its best shot?" —

their fork, their guards, their config, perfect intrinsics, and a harness we debugged in its favor — the articulation, scale, speed and in-the-wild gaps are the method's own.

The seven-method ladder — and the sharpest single finding of the baseline campaign. With raw per-frame HaMeR scored (no temporal fit, no flip guard: RA 19.0 / PA 7.8 / jerk 9.6) the full ladder on the 6 GT windows reads: frame-independent HaMeR (19.0/7.8/9.6) → + our temporal opt (16.2/7.8/3.4) → sequence-native direct-MANO head (10.8/5.8/2.6) — each rung of added temporal structure buys a discrete jump, the empirical curve of the sequence-first thesis. And one number indicts the batch-optimization approach outright: Dyn-HaMR's own initialization IS raw HaMeR at PA 7.8 mm — after its joint hands+camera optimization, PA lands at 26.8 mm. Its optimizer trades articulation away to satisfy camera and reprojection constraints; ours never lets that trade happen. Source: results_rawhamer.json.

Backend and lineage invariance (added after the DA3 + upstream runs). Three more baseline regimes close the question of "did we compare against the right Dyn-HaMR": the somantis fork's

latest production backend (DA3-Streaming, pinned GT K) scores identically to the ViPE-pin run (PA 26.8 vs 26.8, RA 33.9 vs 34.0 — the backend changes speed, not accuracy), and the original upstream Dyn-HaMR at its own defaults (DROID est-K) is worse than the somantis fork on every global axis (G 57 vs 40 mm, absolute 194 vs 43 mm, jerk 13.7 vs 9.4 — its estimated focals swing to 6.8× the true value on ego footage). Our articulation and root-relative margins are invariant across all four Dyn regimes. Source: results_dyn_da3.json, results_dyn_upstream.json.

Seeing the strongest-baseline run. The table above, made visible. All panels below use the

one shared renderer (§5.2 conventions — same MANO mesh material, same AR projection, same follow-orbit camera): TOP = ours (champion v11, full production pipeline), BOTTOM = the production-hardened Dyn-HaMR fork consuming pinned exact GT K with all its guards active and the corrected camera loader — the baseline at its genuine best. Both AR overlays now sit properly on the hands; the differences live in the orbit panel (hand scale, steadiness) and in the table (articulation 3.4×, jerk ~3×). These side-by-sides are also embedded directly in the

model gallery.

5.5 2D-tracking baseline — MediaPipe Hands, head-to-head

MediaPipe Hands is the industry-default 2D hand tracker, so we score it as a formal baseline on both of our 2D instruments. Caveat first, as always: MP pseudo-labels are part of our wild training mix (it is partially our teacher), but the GT rows below are scored against exact GT-2D projections on the 27 frozen lab windows — an instrument neither method was fit to — and the stability rows are pure geometry. MP hand→slot assignment is deliberately generous (both handedness conventions tried per clip, best kept).

GT-2D accuracy (27 windows, PCK normalized by GT hand-bbox diagonal):

Wild stability (same 6-clip instrument and formulas as our promotion gate):

We lead broadly — whole-hand accuracy +13–18 PCK points with perfect recall, jitter ~2.8× lower, flicker 13× lower at a 25-point higher present rate. Two honest reads the table also forces: (1) MP's lower teleport count is an absence artifact — teleports are only countable inside present spans, and MP simply drops the hand on the frames where our tracker holds and occasionally spikes; a tracker that is absent 26% of the time cannot teleport. (2) MediaPipe beats us on ARCTIC fingertips (tip-PCK 0.270 vs 0.187): its crop-refinement pipeline preserves fingertip spread at high resolution where our 512-letterbox Stage-1 under-spreads — independent confirmation of the uniform span-shrink we isolated in the depth audit (§5.3), and exactly the axis the v15 local-head experiment targets. Source: local_2d/gt3d_eval/results_2d_baseline_mp.json + local_2d/wild_stability/mp_baseline.json.

Takeaway (corrected 2026-07-04). We found and fixed a harness bug that had been crippling
the baseline, re-ran everything, and the verdict survives — smaller in places, sharper
everywhere, and now fairly won. On exact GT with the corrected runs (§5.3/§5.4, same-support 6
windows, ours = v11 champion): articulation PA-Sim(3) 8.0 vs 26.8 mm (3.4×) and **SE(3)
9.0 vs 27.2 mm (3.0×)** — invariant across bugged/fixed, est-K/perfect-K, so it is the method's
own; root-aligned 16.3 vs 34.0 mm (2.1×); reprojection 0.044 vs 0.163 (3.6×); jerk
3.5 vs 9.4 mm/f³; hand length +0.3 vs −6.3 cm; runtime **~0.4 s/frame vs ~3 min/clip
. What the corrected baseline wins, we print in bold too: with pinned exact GT K** it
converges 6/6 and its absolute depth beats ours (mean |Δ| 29 vs 71 mm) — our +8 cm far
bias is now the top roadmap item this comparison produces — and it never drops a track
(ours: one 2-frame dropout in 6,480). But perfect K is a lab-only gift: with its own estimated
camera its absolute error is 187 mm median with +1.4 m blow-ups, and on wild footage it still
converges on only 2/8 clips (one negative-focal divergence, one zero-track clip). The picture
is coherent and honest: **a per-sequence optimizer given a perfect metric camera places hands
in depth better than we do; on every axis that measures hand pose quality — articulation,
stability, scale-sanity, speed, wild robustness — the feed-forward pipeline leads by 2–4×,
under a harness we debugged in the baseline's favor.**