Using the microphone to create unique games

this guide is for game developers using airconsole 1 11 0 js inside their game use airconsole getusermedia() when a controller needs microphone access, for example for voice volume, singing, blowing, rhythm, or other audio reactive gameplay quick start add a button to your controller html enable microphone then wait until airconsole is ready and request microphone access from that button in most games, requesting from an intentional player action is clearer than asking immediately on page load and various browsers require user interaction for permission requests to work var micstream = null; function setmicrophonestatus(message) { document getelementbyid("microphone status") textcontent = message; } airconsole onready = function() { document getelementbyid("request microphone") onclick = function() { airconsole getusermedia({ audio true }) then(function(stream) { micstream = stream; setmicrophonestatus("microphone enabled"); // use the stream, for example with the web audio api }) catch(function(error) { // show a useful message or keep the game playable without microphone input setmicrophonestatus("microphone is not available"); console error("microphone access failed ", error); }); }; }; stop the microphone when you no longer need it if (micstream) { micstream gettracks() foreach(function(track) { track stop(); }); micstream = null; } stopping tracks matters it releases the microphone for the browser or app and lets privacy indicators turn off when nothing else is using the microphone current limits as of 2026 05 19, the airconsole api 1 11 0 supports getusermedia only on controllers and it only supports the audio constraint, not video // rejects in airconsole 1 11 0 airconsole getusermedia({ video true }); // rejects in airconsole 1 11 0 on the screen airconsole getusermedia({ audio true }); what you can request in airconsole 1 11 0, you can call airconsole getusermedia({ audio true }); to request access to the users audio stream on the controller you can also request user media constraints for audio airconsole getusermedia({ audio { echocancellation true, noisesuppression true, autogaincontrol true } }); where it works after airconsole onready has fired, call getusermedia() from a controller, not from the screen calling before onready , before the controller has a device id, rejects getusermedia() with notready the screen can learn whether one of the controllers was granted or denied microphone access through callbacks airconsole onusermediaaccessgranted = function(deviceid) { console log("controller " + deviceid + " has microphone access"); }; airconsole onusermediaaccessdenied = function(deviceid) { console log("controller " + deviceid + " was denied microphone access"); }; the controller that requested access should use the returned promise to know its own result airconsole getusermedia({ audio true }) then(function(stream) { // this controller got access }) catch(function(error) { // this controller did not get access }); handling errors getusermedia() returns a promise it rejects when airconsole cannot start the request or when the browser/app cannot grant microphone access airconsole specific errors extend airconsoleusermediaerror error name === "airconsole usermediaerror" common airconsole specific messages notsupportedonscreen the screen called getusermedia() notready airconsole is not ready yet alreadypending another microphone request is already in progress on this controller invalidconstraints the constraints are missing, false, or unsupported by airconsole 1 11 0 timeout the airconsole permission flow timed out permissiondenied airconsole/platform permission flow denied the request browser originated errors use the browser's own error names common ones are notallowederror the user, browser, page, iframe policy, or app settings denied microphone access notfounderror no matching microphone was found notreadableerror the microphone exists, but the browser/os could not open it overconstrainederror your audio constraints could not be satisfied aborterror the browser could not use the device after starting the request a practical handler function showmessage(message) { document getelementbyid("microphone status") textcontent = message; } function handlemicrophoneerror(error) { if (error instanceof airconsoleusermediaerror) { if (error message === airconsole user media error type notsupportedonscreen) { showmessage("microphone input is only available on controllers "); } else if (error message === airconsole user media error type alreadypending) { showmessage("microphone permission is already being requested "); } else if (error message === airconsole user media error type timeout) { showmessage("microphone permission timed out try again "); } else { showmessage("microphone is not available right now "); } return; } if (error instanceof domexception) { if (error name === "notallowederror") { showmessage("allow microphone access in your browser or app settings "); } else if (error name === "notfounderror") { showmessage("no microphone was found on this device "); } else if (error name === "overconstrainederror") { showmessage("this microphone does not support the requested audio settings "); } else { showmessage("could not start the microphone "); } return; } showmessage("could not start the microphone "); } recommended game flow ask for microphone access only when the player intentionally enables an audio feature a button in the controller lobby is usually better than requesting immediately on page load as certain browsers will reject permission requests without user interaction if you use the request microphone button from the quick start html above, the flow is airconsole onready = function() { document getelementbyid("request microphone") onclick = function() { airconsole getusermedia({ audio true }) then(startmicrophonegameplay) catch(handlemicrophoneerror); }; }; keep the game playable when microphone access fails for example, keep button controls available or let the player retry do not call getusermedia() repeatedly while one request is pending airconsole rejects concurrent requests with alreadypending do as much of the audio processing as possible on the controller the message rate limit and the users network bandwidth can impact the users game experience otherwise example send audio level to the screen this example measures microphone volume on the controller and sends a simple level value to the screen controller html enable microphone stop microphone controller javascript var micstream = null; var audiocontext = null; var analyser = null; var animationframe = null; var lastsendtime = 0; var source = null; airconsole onready = function() { document getelementbyid("request microphone") onclick = startmiclevelinput; document getelementbyid("stop microphone") onclick = stopmiclevelinput; }; function setmicrophonestatus(message) { document getelementbyid("microphone status") textcontent = message; } function startmiclevelinput() { airconsole getusermedia({ audio true }) then(function(stream) { var audiocontextclass = window\ audiocontext || window\ webkitaudiocontext; micstream = stream; audiocontext = new audiocontextclass(); source = audiocontext createmediastreamsource(stream); analyser = audiocontext createanalyser(); analyser fftsize = 256; source connect(analyser); setmicrophonestatus("microphone enabled"); sendmiclevelloop(); }) catch(handlemicrophoneerror); } function sendmiclevelloop() { var data = new uint8array(analyser frequencybincount); function tick() { animationframe = requestanimationframe(tick); var now = date now(); // we want to send the information only 10 times per second of our 25 messages / second limit if (now lastsendtime < 100) { return; } lastsendtime = now; analyser getbytefrequencydata(data); var sum = 0; for (var i = 0; i < data length; i++) { sum += data\[i] data\[i]; } var level = math sqrt(sum / data length) / 255; airconsole message(airconsole screen, { type "audio level", level level }); } tick(); } function stopmiclevelinput() { if (animationframe) { cancelanimationframe(animationframe); animationframe = null; } if (micstream) { micstream gettracks() foreach(function(track) { track stop(); }); micstream = null; } if (source) { source disconnect(); source = null; } if (analyser) { analyser disconnect(); analyser = null; } if (audiocontext) { audiocontext close(); audiocontext = null; } setmicrophonestatus("microphone stopped"); } function handlemicrophoneerror(error) { setmicrophonestatus("microphone is not available"); console error("microphone access failed ", error); } screen airconsole onmessage = function(deviceid, data) { if (data && data type === "audio level") { var level = data level; // use level for gameplay or visualization } }; the example sends at most 10 audio level messages per second keep audio telemetry small and rate limited; games should not stream raw microphone data through airconsole messages for a complete game structure, start from the airconsole pong example repository https //github com/airconsole/games pong https //github com/airconsole/games pong in this game you will find controller html which implements the getusermedia requesting additionally you will find screen html that subscribes to the airconsole onusermediapermissiongranted https //airconsole github io/airconsole api/beta/airconsole microphone permission html#onusermediaaccessgranted and airconsole onusermediapermissiondenied https //airconsole github io/airconsole api/beta/airconsole microphone permission html#onusermediaaccessdenied callbacks use those as full context examples for where controller setup, onready , lobby ui, screen messages, and microphone cleanup live in an actual game browser and app behavior to expect browsers and mobile apps do not expose microphone permission exactly the same way on web controllers the browser may show its own microphone prompt the promise can stay pending if the player ignores the prompt airconsole rejects with timeout after its own timeout period browser failures such as notallowederror or notfounderror may be returned on native airconsole controller apps the app may show native permission ui if permission was denied permanently, the player may need to open system settings airconsole will inform the user about that older app versions may not support the feature airconsole will in this case inform the user about the need to update the application build your game ui around outcomes, not platform assumptions granted start audio gameplay denied when the user denies microphone access, the platform will be handling explain how to allow microphone access or offer a retry checklist call getusermedia() only from controllers request microphone with { audio true } unless you have a reason for advanced audio constraints handle both airconsole usermediaerror and browser error names keep a non microphone fallback stop all tracks when leaving the audio feature, lobby, or game rate limit any audio derived messages sent to the screen do not request video through airconsole 1 11 0 references mdn mediadevices getusermedia() https //developer mozilla org/en us/docs/web/api/mediadevices/getusermedia https //developer mozilla org/en us/docs/web/api/mediadevices/getusermedia mdn mediatrackconstraints https //developer mozilla org/en us/docs/web/api/mediatrackconstraints https //developer mozilla org/en us/docs/web/api/mediatrackconstraints mdn mediastreamtrack stop() https //developer mozilla org/en us/docs/web/api/mediastreamtrack/stop https //developer mozilla org/en us/docs/web/api/mediastreamtrack/stop