By name: xb-sound
description: >-
Play and capture audio in an XR Blocks app via xb.sound — spatial (positional)
audio anchored to scene objects, master volume control, microphone recording and
playback, plus speech recognition and synthesis. Use to add sound effects,
3D-positioned audio, voice recording, or text-to-speech / speech-to-text. Covers
xb.core.sound (startRecording/stopRecording/playRecordedAudio,
getAudioListener, setMasterVolume) and the exported SpeechRecognizer /
SpeechSynthesizer / SoundSynthesizer classes. See samples/sound.
xb-sound: spatial audio & speech
xb.sound (a.k.a. xb.core.sound) manages audio. See samples/sound, demos/sound_detector,
and src/sound/ (which has its own README).
Spatial (positional) audio
Attach a THREE.PositionalAudio to any object so it pans/attenuates with the listener:
const listener = xb.core.sound.getAudioListener();
const audio = new THREE.PositionalAudio(listener);
audio.setBuffer(audioBuffer); // a decoded AudioBuffer
audio.setRefDistance(0.5);
audio.setVolume(1.0);
ball.add(audio); // parented to the object → 3D positioned
audio.play();
Microphone recording & playback
await xb.core.sound.startRecording();
// …later…
const pcm = xb.core.sound.stopRecording(); // ArrayBuffer of Int16 PCM
const rate = xb.core.sound.getRecordingSampleRate();
await xb.core.sound.playRecordedAudio(pcm, rate);
xb.core.sound.setMasterVolume(0.8); // 0..1
Speech recognition & synthesis
The SDK exports xb.SpeechRecognizer and xb.SpeechSynthesizer (plus xb.SoundSynthesizer
and xb.BackgroundMusic). Use them for speech-to-text and text-to-speech; see
src/sound/SpeechRecognizer.ts and
src/sound/SpeechSynthesizer.ts for their APIs.
Notes
- Microphone access prompts for permission; set
options.permissions.microphone = true(or use a feature that requests it) before relying on recording. - Recorded data is Int16 PCM; convert to a
THREEAudioBufferfor positional playback (see the conversion insamples/sound/main.js). - Call
xb.core.sound.disableAudio()to tear down capture when done.