Javascript Sound Experiments - Spectrum Visualization [ProcessingJS + WebAudio]
This entry is just a quick add-on for my ongoing sound related project that maybe worth sharing. I came up with the need of visualizing sound spectrum. You might find a dozen of tutorials on the topic, but I didn’t think that it is so easy to do with processingJS & Chrome’s WebAudio Api. I found a brilliant and simple guide on the web audio analyser node here . This is about to make the visuals with native canvas animation technique. I used the few functions from this tutorial and made a processingJS port of it. The structure of the system is similar to my previous post on filtering:
1. Load & playback the sound file with AudioContext (javascript)
2. Use a function to bridge native javascript to processingJS (pass the spectrum data as an array)
3. Visualize the result on the canvas
Load & playback the sound file with AudioContext (javascript)
We are using a simpler version of audio cooking within this session compared to the previous post. The SoundManager.js file simply prepares our audio context, loads and plays the specified sound file once the following functions are called:
initSounds();
startSound('ShortWave.mp3');
We call these functions when the document is loaded. That is all about sound playback, now we have the raw audio data in the speakers, let’s analyze it.
Use a function to bridge native javascript to processingJS
There is a function to pass the spectrum data as an array, placed in the head section of our index.html:
function getSpektra()
{
// New typed array for the raw frequency data
var freqData = new Uint8Array(analyser.frequencyBinCount);
// Put the raw frequency into the newly created array
analyser.getByteFrequencyData(freqData);
var pjs = Processing.getInstanceById('Spectrum');
if(pjs!=null)
{
pjs.drawSpektra(freqData);
}
}
This function is based on the built-in analyzer that can be used with the WebAudio API. the “freqData” object is an array that will contain all the spectral information we need (1024 spectral bins by default) to be used later on. To find out more on the background of spectrum analysis, such as Fourier transformation, spectral bins, and other spectrum-related terms, Wikipedia has a brief introduction.
However we don’t need to know too much about the detailed backgrounds. In our array, the numbers are representing the energy of the sounds from the lowest pitch to the highest pitch in 1024 discrete steps, all the time we call this function (it is a linear spectral distribution by default). We use this array and pass to our processing sketch within the getSpectra() function.
Visualize the result on the canvas
From within the processing code, we can call this method any time we need. We just have to create a JavaScript interface to be able to call any function outside of our sketch (as shown in the previous post, also):
interface JavaScript
{
void getSpektra();
}
// make the connection (passing as the parameter) with javascript
void bindJavascript(JavaScript js)
{
javascript = js;
}
// instantiate javascript interface
JavaScript javascript;
That is all. Now we can use our array that contains all the spectral information by calling the getSpectra() function from our sketch:
// error checking
if(javascript!=null)
{
// control function for sound analysis
javascript.getSpektra();
}
We do it in the draw function. The getSpectra() is passing the array to processing in every frame, so we can create our realtime visualization:
// function to use the analyzed array
void drawSpektra(int[] sp)
{
// your nice visualization comes here...
fill(0);
for (int i=0; i<sp.length; i++)
{
rect(i,height-30,width/sp.length,-sp[i]/2);
}
}
As usual, you can download the whole source code from here as well as try out before using it with the live demo here. Please note, the demo currently works with the latest stable version of Google Chrome.