PoseDetection Interface v2

Pose Detection Interface brings universal human pose detection to Unity projects using the industry-standard COCO pose keypoint format. Works with any COCO pose-segmentation model — v2 adds a full multi-person tracking pipeline, letterbox-correct preprocessing, OKS-based duplicate suppression, offline replay, Timeline integration, and a virtual gesture input device.

Universal Model Support:

- Compatible with ANY COCO pose-segmentation keypoint format model

- Auto-detects End-to-End [1, N, 57] and Anchor-based [1, 56, N] output layouts

- Pre-configured for all 17 COCO standard keypoints

- Three declarative normalization modes (Unit01, Byte255, ImageNet)

- Example ONNX models included

- One-click model swap from the Inspector; hot-swap at runtime without stale callbacks

Accuracy & Tracking (new in v2):

- Letterbox preprocessing with gray padding — correct results on non-square cameras

- Inverse coordinate mapping so keypoints always come back in source-frame pixels

- Test-time horizontal-flip augmentation with anatomical left/right swap

- Per-keypoint confidence thresholds

- OKS (Object Keypoint Similarity) NMS using COCO sigmas

- Multi-person tracking via Hungarian matching with augmented cost (torso distance + bbox-IoU + OKS + velocity)

- Constant-velocity predictor so tracks survive short occlusions

- Tentative → Confirmed state machine for flicker-free tracking IDs

- Plug-in interface for custom appearance-based re-identification

Key Applications:

- Fitness and workout tracking (squat counter included)

- Interactive games with body movement input

- Motion capture and pose-driven animation

- AR/VR body tracking

- Gesture-driven UI (virtual Input System device — bind gestures like any button)

- Deterministic replays for QA, cutscenes, and Timeline-driven demos

Fully Customizable:

- Complete source code included for custom implementations

- Modular architecture — swap input sources, inference, parsing, smoothing, or visualizer independently

- Extensive configuration exposed in the Inspector for performance tuning

- Runtime (and Editor) assembly definitions for fast compile times

- Profiler markers on every pipeline stage for targeted optimization

- Cross-platform deployment including Android

Tooling (new in v2):

- One-click editor menu scaffolds under GameObject → Pose Detection (auto-assigns a bundled model)

- Offline PoseReplayer — play back recorded JSON without a camera or model

- Timeline track + clip for deterministic pose playback

- Gesture Input Driver — expose T-pose, hands-up, arms-crossed, wave, and more as Input System actions

- Pose Live Comparison component for reference-clip matching

- PoseRecorder with in-editor scrub / step / speed UI

Content Includes:

- Example ONNX models (PoseNet-v2 family: Nano, Small, Medium, Large, XLarge)

- Seven example scenes: Real-time Detection, Video Input, Squat Counter, Pose Recorder, Pose Replayer, Gesture Detector, Stance Estimator

- Ready-to-use prefabs for drag-and-drop integration

- Comprehensive scripting API for custom model integration

Made by Yasin Shabani Varaki aka MrFresheyPAX