QuestVisionKit 是一个包含模板和参考项目的集合,展示了如何使用 Meta Quest 的新型 Passthrough Camera API 进行高级 AR/VR 视觉处理,
QuestVisionKit 是一个包含模板和参考项目的集合,展示了如何使用 Meta Quest 的新型 Passthrough Camera API 进行高级 AR/VR 视觉处理,
Build mixed reality experiences that can see the world around you.
Unity samples for Meta Quest 3 and Quest 3S: sample real-world colors, detect objects, track QR codes, create camera-based shader effects, ask questions about your surroundings, and stream video to another device.
Quick start · Samples · Troubleshooting · Builds & testing · Community showcase
Use a headset build to test the camera workflows. An Editor preview does not establish on-device camera or tracking behavior. See Meta’s Passthrough Camera API documentation for platform requirements.
git lfs install
git clone https://github.com/xrdevrob/QuestCameraKit.git
cd QuestCameraKit
git lfs pullUnity-QuestVisionKit in Unity Hub with the editor version above.The build includes all six headset samples, starting with ColorPicker. The desktop WebRTC receiver is separate.
| Action | Quest controller | Editor keyboard |
|---|---|---|
| Show or hide menu | Y | M |
| Choose sample | Left thumbstick up/down | ↑ / ↓ |
| Open sample | X | Enter |
Scenes live in Unity-QuestVisionKit/Assets/Samples.
| Sample | Explore | Scene |
|---|---|---|
| Color picker | Map a point in the room to a camera pixel | ColorPicker |
| Object detection | Run inference and place detection markers in 3D | ObjectDetection |
| Native QR tracking | Read QR payloads and display spatial bounds with Meta MRUK | QRCodeDetection |
| Camera shaders | Stereo camera mapping, frosted glass, and portal effects | CameraMappingForShaders |
| Image + voice AI | Ask a spoken question about a camera image | ImageLLM |
| WebRTC streaming | Send camera video to a receiving peer | WebRTC-Quest |
The demos below illustrate the samples; they were captured with earlier project versions.
Aim at a surface and press/release A to sample its color. The sample uses environment raycasting to locate a point in the room, projects it into the camera image, and applies the sampled color to a virtual object.
Run the bundled YOLO model with Unity Inference Engine and project its detections into the room. Detection runs automatically. Configure Label Filters on ObjectRenderer to show selected classes, or leave the list empty to show all model classes. Confidence filtering applies when the model supplies scores.
See the supported labels. Detection accuracy and performance depend on the scene, model, and device.
Detect QR codes through Meta MRUK, with no additional decoder or NuGet setup. The sample displays the decoded payload and the bounds supplied by native tracking.
Use the controller-mounted panel to request Scene / spatial-data permission, check support, and enable tracking. Present a clear, well-lit QR code. Native tracking updates spatial poses over time; it is not intended for fast-moving object tracking. See Meta’s QR tracking guide for requirements and limitations.
Explore stereo passthrough mapping, frosted glass, and wavy portal materials in one scene. The sample uses both camera feeds and exposes per-eye UV offsets for calibration.
Record a question, transcribe it, send the text and a camera frame to OpenAI, and play the response with text-to-speech.
ImageOpenAIConnector / OpenAI Manager.Keep real keys out of commits and distributed APKs. For a shipped application, route authenticated requests through your own backend. This sample requires an OpenAI account with API access; network requests may incur charges.
Stream camera video using SimpleWebRTC, Unity WebRTC, and NativeWebSocket.
WebRTC-Quest, the connection is under [BuildingBlock] Camera Rig/TrackingSpace/CenterEyeAnchor/Client-STUNConnection.WebRTC-Quest on the headset and open WebRTC-SingleClient as the receiving peer in Unity.See the signaling setup tutorial. Both peers must be able to reach the signaling server; successful signaling alone does not prove video is flowing.
The project pins its dependencies in manifest.json and packages-lock.json.
| Component | Pinned version |
|---|---|
| Unity Editor | 6000.3.12f1 |
| Meta XR Core / MRUK | 205.0.0 |
| Unity Inference Engine | 2.6.1 |
| OpenXR / Meta OpenXR | 1.18.0 / 2.6.1 |
| Universal Render Pipeline | 17.3.0 |
| Unity WebRTC | 3.0.0 |
| Quak device testing | 0.4.0-alpha.1 |
SimpleWebRTC and NativeWebSocket are pinned to Git revisions. See third-party dependencies for provenance and licensing.
The testing guide covers individual and combined APK builds, Unity regression checks, scene audits, and the included Quak device tests. Quak is the renamed XRQA project.
For a quick repository check:
python3 scripts/check_repo.py
git diff --checkThe September 2026 maintenance pass built the combined app and all six individual Android ARM64 IL2CPP samples, passed Unity regression and missing-script checks, and passed GitHub repository checks and CodeQL. On-device acceptance remains pending: the available Quest 3S session was blocked by the system sensor lock. ImageLLM cloud requests and WebRTC peer streaming also require their external setup and validation.
| Symptom | Check |
|---|---|
| Camera feed does not start | Keep the headset awake and unlocked, accept permissions, and confirm you are running a headset build. |
SensorLockActivity blocks launch |
Put on the headset and press its physical Power button once to clear the lock, then retry. |
| QR codes do not appear | Check the native panel’s supported/enabled state and spatial-data permission; use a clear QR code in good light. |
| Packages fail to resolve | Use the pinned Unity editor; verify network/Git access and let Package Manager finish. |
| Assets are missing after cloning | Run git lfs pull, then the repository check above. |
| ImageLLM does not respond | Check the private API key, model access, microphone permission, and network connection. |
| WebRTC does not connect | Configure both peers, enable their signaling connections, and confirm the server is reachable. |
| Non-development build rejects Quak | Disable its OpenXR injection layer as described in the testing guide. |
Explore the community showcase for tutorials, experiments, and projects built with Quest camera access. Contributions are welcome—see CONTRIBUTING.md and the code of conduct.
Report an issue · Join Discord · Follow on X · LinkedIn
Please follow the media-hosting guide when adding demo images or videos.
QuestCameraKit is MIT licensed. Included third-party code and packages retain their own licenses; see third-party dependencies. A credit or link back is always appreciated.
暂无开放 Issues,或尚未同步最近议题。