View on Asset Store (opens in new tab)
Package details
- Latest version
- 0.1.1
- Last updated
- First released
- Unity version
- 6000.3.9 or later
- Package size
- 1008 KB
- Files
- 76
- Favorites
- 1
- Views in the past week
- 9
Price tracking
- Last checked
- Tracked since
- Lowest recorded
- $4.99 on
Keywords
Perception for ML-Agents
- Price drop
A toolkit for generating computer vision ground truth (Bounding Box, Semantic Segmentation, Depth) and integrating it with ML-Agents as observations for reinforcement learning training.
Lowest recorded price
Get 44% off on Asset Store (opens in new tab)Affiliate link · Purchases support this site
Price history
Last 6 months
USDAsset preview
1 / 3
Perception for ML-Agents screenshot 1
About this asset
Perception is designed to empower Unity ML-Agents with advanced computer vision capabilities. It allows a standard Unity Camera to generate rich Ground Truth data including 2D Bounding Boxes, Semantic Segmentation masks, and Gray-scale Depth maps.
Perception is specifically engineered for runtime integration with Unity ML-Agents. It wraps these ground truth generators into custom Sensor Components, allowing Agents to directly observe the segmented labeled, and depth world. The observations are serialized and sent to the external Python environment via ML-Agents API, enabling the training of complex vision-based reinforcement learning models. The Python samples is available on GitHub https://github.com/BlueFisher/Perception-for-ML-Agents-Sample/blob/master/Perception_unity_wrapper.ipynb.
Perception is a fork of the now-deprecated com.unity.perception. It strips out the dataset capture (offline data collection) features to focus on lightweight, real-time ground truth generation and has been migrated to support Unity 6 render graph.
Require com.unity.ml-agents >= 4.0.0 (ML-Agents Release 23)
From the publisher's Asset Store listing.
Technical details
Perception Capabilities
- Bounding Box Generation: Real-time calculation of 2D bounding boxes for labeled game objects (BoundingBoxPerceptionCamera).
- Semantic Segmentation: Pixel-perfect object classification rendered to textures (SemanticSegmentationPerceptionCamera).
- Depth Render: Grayscale depth map rendering (DepthRenderPerceptionCamera).
ML-Agents Integration
- Custom Sensors: Ready-to-use components that attach to Agents:
- BoundingBoxComponent: Feeds bounding box coordinates as Buffer Sensor observations.
- SemanticSegmentationComponent: Feeds segmentation masks as Render Texture observations.
- DepthRenderComponent: Feeds depth maps as Render Texture observations.
Architecture & Tools
- Labeling System: Robust configuration via LabelConfig and IdLabelConfig to manage object classes and IDs.
- URP Support: Utilizes PerceptionRendererFeature and custom render passes for efficient ground truth generation.
Compatibility
| Unity version | Built-in | URP | HDRP |
|---|---|---|---|
| 6000.3.9f1 | Not listed | Compatible | Not listed |
Require Unity 6 render graph
As listed on the Asset Store.