Imperello Vision

Imperello Vision adds live face, pose, hand, and gesture tracking to your Unity project using on-device ML. It runs on the player's device, no cloud and no API keys.

Use it for avatars, UI, fitness games, gesture controls, object-aware gameplay, background effects, stylized portraits, and similar features.

What you can do

• Face tracking - follow facial points for avatars or effects
• Pose tracking - follow body joints for movement and games
• Hand tracking - follow hand points for interaction
• Gesture recognition - detect gestures like thumbs up or open palm
• Object detection - find objects in the webcam or your own video file
• Image classification - label the whole frame (for example everyday objects)
• Face bounds detection - lightweight face boxes
• Image segmentation - person/selfie mask for background removal and effects
• Face stylizer - turn a face capture into a stylized avatar-style look
• Holistic landmarker - face, pose, and hands together on one frame
• Fitness Trainer sample - pose-based rep counter for Squats, Jumping Jacks etc
• Situation Aware sample - live person / object / gesture context for avatars and local AI
• Background Replace sample - virtual green screen (Solid, Studio, Blur, or your own video)

Fitness Trainer sample

Ship a workout-style experience without building pose logic from scratch:

• Counts Squats, Jumping Jacks, and Arm Raises from a live webcam
• On-screen coach guide, rep goal, timer, and Pause / Resume / Reset / Finish
• Full-body tracking check so out-of-frame guesses do not falsely complete
• Editable sample scripts you can copy into your own fitness game
• Open via Tools → Imperello Vision → Open Fitness Trainer Sample

Situation Aware sample

• Describe what a person is doing and holding, for avatars and local AI:
• Combines Holistic (face + pose + hands) with optional Gesture and Object Detection
• Produces a short situation sentence plus structured fields you can poll in C#
• Copy ImperelloSituationAwareness.cs into your project
• See Documentation/SituationAware.md
• Open via Tools → Imperello Vision → Open Situation Aware Sample

Background Replace sample (new in 1.3.1)

• Virtual green screen from the selfie segmenter:
• Modes: Solid, Studio, Blur, and Video (buyer-supplied VideoClip or StreamingAssets MP4)
• Soft matte edges with Loop Video on by default
• Copy ImperelloBackgroundReplace.cs into your project
• See Documentation/BackgroundReplace.md
• Open via Tools → Imperello Vision → Open Background Replace Sample

How to start

1. Import the package into Unity.
2. Open a sample scene (Face, Pose, Hand, or Gesture).
3. Press Play with a webcam connected.

Optional: use Tools → Imperello Vision → Setup Tools to confirm plugins and models are OK, or to open samples from one place.

More help: Documentation/ImportGuide.md

Code examples: Documentation/GettingStarted.md

AI Skills

AI Skills Imperello Vision includes project AI Skills under AI/Skills/ to help you set up and integrate faster with AI agents:

• Works with Unity AI Assistant (Unity 6 → Edit → Preferences → AI → Skills)
• Compatible with coding agents that read your project (Cursor, Claude, GitHub Copilot, Windsurf, and similar)
• Skills cover setup, all ten vision features, the Fitness Trainer showcase and Background Replace

What's included

• Imperello libraries ready to use
• Native plugins for Windows and Mac
• Model files for all supported vision features
• 10 editable demo scenes with short in-scene help overlays
• Editable demo scenes with short in-scene help overlays

Platforms

Windows - Editor and built games

Mac - Editor and built games

Requirements

• Unity 6.0 or newer (tested on 6.3 LTS)
• Webcam for live demos
• Tested on Windows 64-bit and Mac (Apple Silicon)

You can use this in commercial Unity projects. Open-source parts are licensed under Apache 2.0 and similar terms; see Third-Party Notices.txt in the package.