PUBLIC MARKS with tags sounds & html5

Here is Beez, a web real-time audio experiment using smartphones as synthesizer effect controllers. This is our second Web Audio API experiment made in one Hackday at Zenexity. This time, we were much more focused on having the best latency performance: we used the bleeding-edge WebRTC technology, which allows you to link clients in Peer-to-Peer instead of a classical Client-Server architecture.

A few years ago, I wrote a little ActionScript 3 library called MicRecorder, which allowed you to record the microphone input and export it to a .WAV file. Very simple, but pretty handy. The other day I thought it would be cool to port it to JavaScript. I realized quickly that it is not as easy. In Flash, the SampleDataEvent directly provides the byte stream  PCM samples) from the microphone. With getUserMedia, the Web Audio APIs are required to extract the samples. Note that getUserMedia and Web Audio are not broadly supported yet, but it is coming. Firefox has also landed Web Audio recently, which is great news.

WebAudio Waveform Visualizer Customizable waveform audio visualzation built on top of WebAdio and HTML5 Canvas (plus, optional SVG renderer). With wavesurfer.js you can assemble a full-featured HTML5 audio player or create a sophisticated DJ application.

This project uses the new HTML5 Audio Data API. At the moment, only Firefox 4 beta supports this. Click the visualization area to toggle "fullscreen" mode.

If you've read some of my other articles on this blog you probably know I'm a fan of HTML5. With HTML5 we get all this interesting functionality, directly in the browser, in a way that, eventually, is standard across browsers. One of the new HTML5 APIs that is slowly moving through the standardization process is the Web Audio API. With this API, currently only supported in Chrome and as of October 2013 also in Firefox, we get access to all kinds of interesting audio components you can use to create, modify and visualize sounds (such as the following spectrogram).

ThreeAudio helps you create music visualizations in Three.js, by exposing audio data in GLSL shaders. It can be used directly with Three.js or as a tQuery plug-in.

Enhance your products with live, face-to-face communication.

PeerJS wraps the browser's WebRTC implementation to provide a complete, configurable, and easy-to-use peer-to-peer connection API. Equipped with nothing but an ID, a peer can create a P2P data or media stream connection to a remote peer.

You can build cool stuff with WebRTC in five minutes. Want to see what this library can do? Talky.io is powered by SimpleWebRTC! Or checkout the demo (note that screensharing won't work because HTTPS is required.)

Canvas-based UI with visualization options.

Canvas-based UI with visualization options. Note: Spectrum/EQ visualizations

Welcome the my HTML 5 Audio Editor. You can edit WAVE, OGG and MP3 files and export them as WAVE. This application uses no extra plugins like Flash or Silverlight, its written in JavaScript and HTML

Last week Angry Birds for Chrome was updated to use the Web Audio API for all its in-game audio for Chrome users, which means Chrome users get the full Angry Birds experience, without any plugins. The Web Audio API supports a wide variety of use cases, including the high fidelity and low latency requirements of games. Users of other supported browsers will still get sound via Flash or HTML5 audio.

This week, we focused on incorporating sound into the Fieldrunners port. Lucky for us, in-browser audio support has come a long way in the past few years. Given that Google Chrome is a target browser, we were excited to have the opportunity to use the Web Audio API. Before we began, we wanted a clear understanding of what was possible with this new technology and how it compared to HTML5 Audio.

Audio is a huge part of what makes multimedia experiences so compelling. If you've ever tried watching a movie with the sound off, you've probably noticed this. Games are no exception! My fondest video game memories are of the music and sound effects. Now, in many cases nearly two decades after playing my favorites, I still can't get Koji Kondo's Zelda compositions and Matt Uelmen's atmospheric Diablo soundtrack out of my head. The same catchiness applies to sound effects, such as the instantly recognizable unit click responses from Warcraft, and samples from Nintendo's classics. Game audio presents some interesting challenges. To create convincing game music, designers need to adjust to potentially unpredictable game state a player finds themselves in. In practice, parts of the game can go on for an unknown duration, sounds can interact with the environment and mix in complex ways, such as room effects and relative sound positioning. Finally, there can be a large number of sounds playing at once, all of which need to sound good together and render without introducing performance penalties.

This is an open and public initiative to bring more harmony into HTML5 Audio implementations. The specifications are missing some features and sometime leaves room for interpretation. We want to un-mute the Web and make Audio rock! If you feel the same, you can report browsers bugs, write tests, fill issues and join our conversation.

The following illustrate a diverse range of effects which are possible using the convolution engine:

When I started to work on my JavaScript Game Engine back in October 2009, the biggest problems I encountered were with the new HTML5 Audio Element. The Canvas Element already worked nicely in all browsers that supported it at the time, albeit some were a little slow.