meta-xr-sdk

name: meta-xr-sdk description: > Deep integration with Meta's VR/AR developer ecosystem for Quest 2/3/3S/Pro. Covers WebXR (IWER, RATK, @react-three/xr), Meta Spatial SDK (Android), Horizon Platform SDK, hzdb CLI (40+ MCP tools), Unity MCP Extensions, and agentic VR development skills. Use when building immersive WebXR experiences, Quest-native spatial apps, mixed reality with passthrough, hand tracking, spatial anchors, plane/mesh detection, or porting 2D apps to Quest. Complements wasm-js (WASM compute), game-dev (full studio), and unreal-engine (UE5 automation). version: 1.0.0 author: agentbox-skills tags: - meta-quest - webxr - vr - ar - mixed-reality - spatial-computing - hand-tracking - passthrough - react-three-xr - three-js - quest-3 - iwer - ratk - hzdb - spatial-sdk mcp_server: true protocol: stdio entry_point: "npx @meta-quest/hzdb mcp server" env_vars: - ADB_PATH - HZDB_DEVICE_SERIAL compatibility: - "meta-quest-sdk >= 69.0" - "node >= 18" - "@react-three/xr >= 6.0" - "three >= 0.160"

Meta XR SDK

Deep integration with Meta's complete VR/AR developer ecosystem. Bridges WebXR web development, Quest-native Android spatial apps, and AI-assisted agentic workflows via hzdb CLI and MCP tools.

When to Use This Skill

• WebXR immersive experiences targeting Meta Quest browser (Three.js, React Three Fiber)
• Mixed reality with passthrough, plane detection, mesh detection, spatial anchors
• Hand tracking and controller input for VR/AR interactions
• Quest-native spatial apps via Meta Spatial SDK (Android/Kotlin)
• Porting 2D Android apps to Quest spatial environments
• Performance profiling VR apps with Perfetto integration
• Device management (deploy, screenshot, log, shell) via hzdb CLI
• Store submission validation (VRC checks)
• AI-assisted VR development with agentic tools and MCP integration
• Desktop WebXR emulation for development without a physical headset

When Not To Use

• For general 3D modelling/rendering without VR -- use blender
• For Unreal Engine 5 editor automation -- use unreal-engine
• For full game studio orchestration (48 agents) -- use game-dev
• For pure WASM compute graphics without XR -- use wasm-js
• For 3D Gaussian Splatting -- use lichtfeld-studio
• For non-Meta WebXR (generic spec only) -- this skill is Meta-ecosystem-specific

Architecture

+------------------------------------------------------------------+
|                    Meta XR Developer Ecosystem                    |
+------------------------------------------------------------------+
|                                                                    |
|  +-----------------------+    +------------------------------+    |
|  |    Web Platform        |    |    Native Platform            |    |
|  |                        |    |                              |    |
|  |  @react-three/xr       |    |  Meta Spatial SDK            |    |
|  |  Three.js + WebXR API  |    |  (Android/Kotlin)            |    |
|  |  RATK (mixed reality)  |    |  Entity-Component System     |    |
|  |  IWER (emulation)      |    |  Physics, Animation, Input   |    |
|  +-----------+------------+    +-------------+----------------+    |
|              |                               |                     |
|  +-----------v------------+    +-------------v----------------+    |
|  |  WebXR Feature Modules  |    |  Horizon Platform SDK        |    |
|  |                        |    |  (17 API packages)            |    |
|  |  Hand Input Module     |    |  Achievements, IAP, Social    |    |
|  |  Anchors Module        |    |  Leaderboards, DLC, Cloud     |    |
|  |  Hit Test Module       |    |  Storage, Groups, Parties     |    |
|  |  Plane Detection       |    +------------------------------+    |
|  |  Mesh Detection        |                                       |
|  |  AR Module             |    +------------------------------+    |
|  |  Layers API            |    |  hzdb CLI + MCP Server        |    |
|  |  Gamepads Module       |    |  (40+ tools)                  |    |
|  +-----------------------+    |  Device, App, Capture, Perf    |    |
|                                |  Files, Docs, Shell, Logs      |    |
|                                +------------------------------+    |
+------------------------------------------------------------------+
|                    Quest 2 / 3 / 3S / Pro                         |
+------------------------------------------------------------------+

Meta's SDK Ecosystem (Complete Inventory)

1. WebXR Development

IWER -- Immersive Web Emulation Runtime

Desktop WebXR emulation without a physical headset.

npm install iwer @iwer/devui @iwer/sem

Key classes: XRDevice, XRController, XRHandInput

// IWER auto-emulation setup (falls back when no native WebXR)
import { XRDevice } from 'iwer';
import { DevUI } from '@iwer/devui';

const xrDevice = new XRDevice();
xrDevice.installRuntime();
new DevUI(xrDevice);

RATK -- Reality Accelerator Toolkit

Mixed reality utilities bridging WebXR and Three.js.

npm install ratk three

Core class: RealityAccelerator Detected objects: Plane, RMesh, Anchor, HitTestTarget (all extend Object3D)

import { RealityAccelerator } from 'ratk';

const ratk = new RealityAccelerator(renderer.xr);
scene.add(ratk.root);

// Plane detection
ratk.onPlaneAdded = (plane) => {
  plane.material = new MeshBasicMaterial({ wireframe: true });
};

// Spatial anchors
const anchor = await ratk.createAnchor(position, quaternion, true); // persistent

// Hit testing
const hitTarget = ratk.createHitTestTargetFromControllerSpace('right');

// Must call each frame
function animate() {
  ratk.update();
}

API surface:

@react-three/xr -- React Three Fiber XR

Declarative WebXR for React applications.

npm install three @react-three/fiber @react-three/xr@latest

import { Canvas } from '@react-three/fiber';
import { createXRStore, XR } from '@react-three/xr';

const store = createXRStore();

function App() {
  return (
    <>
      <button onClick={() => store.enterVR()}>Enter VR</button>
      <button onClick={() => store.enterAR()}>Enter AR</button>
      <Canvas>
        <XR store={store}>
          <mesh pointerEventsType={{ deny: 'grab' }} onClick={handleClick}>
            <boxGeometry />
            <meshStandardMaterial color="hotpink" />
          </mesh>
        </XR>
      </Canvas>
    </>
  );
}

Capabilities: Hand tracking, controllers, hit testing, anchors, teleportation, DOM overlays, layers, guards, gamepad input, origin management, object detection.

2. Native Spatial Development

Meta Spatial SDK (Android)

Quest-native apps with Android Studio.

Entity-Component System with auto-save DSL:

• Spatial anchors and scene understanding
• Physics (ball socket, cone twist, fixed, hinge, slider, spring constraints)
• Blended animation with delta-time playback
• Mixed reality passthrough integration
• Body tracking, hand tracking
• Spatial video and media playback

Requires: Quest device v69.0+, Android Studio Hedgehog+

Reference apps: Focus, Media View, Geo Voyage, Spatial Scanner

15+ samples: AnimationsSample, BodyTrackingSample, CustomComponentsSample, HybridSample, MediaPlayerSample, MixedRealitySample, MrukSample, Object3DSample, PhysicsSample, SpatialVideoSample, StarterSample, UISetSample, FeatureDevSample

Horizon Platform SDK (17 API packages)

Social platform integration for Quest apps: Achievements, IAP, Social, Leaderboards, DLC, Cloud Storage, Groups, Parties

3. hzdb CLI + MCP Server (40+ Tools)

The Horizon Debug Bridge provides direct device and app management.

# Install and run MCP server
npx @meta-quest/hzdb mcp server

# Or install globally
npm install -g @meta-quest/hzdb

Command groups:

4. Unity MCP Extensions

10 MCP tools for Unity VR/MR development:

Core SDK (3 tools):

• meta_add_camerarig -- VR/MR camera rig setup
• meta_get_config_information -- Configuration retrieval
• meta_update_android_manifest -- Manifest updates

Interaction SDK (7 tools):

• meta_add_canvas_interaction_poke -- UI poke interactions
• meta_add_canvas_interaction_ray -- Distance UI ray interactions
• meta_add_distance_grabbable -- Distance grab mechanics
• meta_add_grabbable -- Standard grab setup
• meta_add_interactionrig -- Interaction rig
• meta_add_teleport_hotspot -- Teleport points
• meta_get_interactors_state -- Interactor status

Requires: Meta Core SDK v78+, Meta Interaction SDK v78+, Unity AI Gateway Beta

5. Agentic VR Development Skills (13 Skills)

Meta's @meta-quest/agentic-tools provides AI-assisted development:

Works with: Claude Code, GitHub Copilot CLI, Cursor

Quick Start Recipes

WebXR + React (Recommended for This Project)

# New project
npm create vite@latest my-xr-app -- --template react-ts
cd my-xr-app
npm install three @react-three/fiber @react-three/xr ratk iwer @iwer/devui

WebXR + Vanilla Three.js

npm init -y
npm install three ratk iwer @iwer/devui webpack webpack-cli webpack-dev-server

Device Management

# Start MCP server for AI-assisted device management
npx @meta-quest/hzdb mcp server

# Direct CLI usage
npx @meta-quest/hzdb device list
npx @meta-quest/hzdb app install ./my-app.apk
npx @meta-quest/hzdb capture screenshot
npx @meta-quest/hzdb perf trace start --duration 10

Mixed Reality (Passthrough + Scene Understanding)

import { RealityAccelerator } from 'ratk';
import { ARButton } from 'ratk';

// Request MR features
const sessionInit = {
  requiredFeatures: ['local-floor', 'hand-tracking'],
  optionalFeatures: ['plane-detection', 'mesh-detection', 'anchors', 'hit-test']
};

document.body.appendChild(ARButton.createButton(renderer, sessionInit));

const ratk = new RealityAccelerator(renderer.xr);
scene.add(ratk.root);

ratk.onPlaneAdded = (plane) => {
  console.log(`Detected: ${plane.semanticLabel} (${plane.orientation})`);
};

ratk.onMeshAdded = (mesh) => {
  mesh.material = new MeshBasicMaterial({ wireframe: true, color: 0x00ff00 });
};

npm Packages Reference

GitHub Repositories

Platform Support

Performance Targets (WebXR on Quest)

Related Skills

References

• Meta Quest Developer Docs
• WebXR Device API Spec
• IWER Documentation
• RATK Documentation
• React Three XR Docs
• Three.js Documentation
• Project Flowerbed Case Study