Visualizers

Visualizers are one of the big new things in iTunes LPs. Click play album and you are presented with nice bobbing visuals to go along with your music. Unfortunately though they aren't really visualizers at all. They're pretty animations but they don't react to the music at all.

Imagine my surprise when I found out the window.iTunes object exposes a waveform property that contains spectrum information about the currently playing track. I'm a bit disappointed that Apple hasn't used this data in their own iTunes LPs. Or at least I haven't seen any that do so, and TuneKit itself has no provisions for using this information.

I don't know all the details about this object yet, but enough to get started. making some really cool things.

The waveform property is an object that contains two properties: waveformData and spectrumData. Both contain arrays of numbers. We will be using the waveformData object. This object contains two properties "0" and "1", however it is strangely not an Array as you would expect. Each of these properties is and Array and as far as I have been able to tell always contain the exact same data. So to get at the waveform data we use window.iTunes.waveform.waveformData[0].

The Array has another peculiarity. The first half of the Array contains a series of values between 0 and 255 indicating the volume level of the corresponding frequency. However the second half of the Array seems to always be filled with zeros.

Now that we know what data is useful to us and how to access it, we need a way to display it. The ideal candidate for such display features is the HTML5 canvas tag which is supported in both Safari and the similar browser in iTunes.

First add something like this to your html. For our example we will use a visualizer of 600x300 resolution:

<canvas id="visualizer" width="600" height="300"></canvas>

Then in your initialization routine set up a drawing loop. A function that is called at regular intervals to redraw the visualizer. It's a good idea to only have that loop running when the visualizer is actually visible ofcourse. And easy way to get such a loop going is using setInterval.

// Call the drawWaveform function every 20 ms (this equals 50 frames per second).
setInterval(drawWaveform, 20);

Now we need to draw something useful onto that canvas every time drawWaveform is called:

function drawWaveform() {
	
	// Get the waveform data
	var theWaveform = window.iTunes.waveform;
	var theWaveformData = theWaveform.waveformData[0];
	
	// Get access to the canvas' 2d drawing context
	var theCanvas = document.getElementById("visualizer");
	var theContext = theCanvas.getContext("2d");
	
	// First we draw an almost transparent black rectangle over the current visual
	theContext.fillStyle = "rgba(0,0,0,0.005)";
	theContext.fillRect(0, 0, 600, 300);
	
	// Then we draw the current image over itself, slightly displaced
	theContext.drawImage(theCanvas, 1, 1, 599, 299, 0, 0, 599, 299);

	// Then set up the colors for the new waveform
	theContext.strokeStyle = "#00cc00";
	theContext.fillStyle = "#004400";
	theContext.lineWidth = 1;
	
	// And begin drawing
	theContext.beginPath();
	theContext.moveTo(200, 300);
	for (var i = 0; i < theWaveformData.length / 2; i++) {
		// We take the waveform data as basis for the height of the wave peaks
		var theHeight = theWaveformData[i];
		theContext.lineTo(200 + 400 * i / (theWaveformData.length / 2), 300 - (theHeight * 50 / 255));
	}
	// Then close the shape
	theContext.lineTo(600, 300);
	theContext.lineTo(200, 300);
	
	// Give it its color and stroke
	theContext.stroke();
	theContext.fill();
	
	// And finish drawing
	theContext.closePath();
	
}

What this does is first draw an almost transparent black rectangle over the existing image, to fade it out a bit. Then it draws itself onto the canvas 1 pixel higher and 1 pixel to the left. Then it draws a filled shape using the waveformdata as basis for the height of the peaks. Which will give an effect like this:

Resources

There are many interesting things you can do with the canvas tag especially in combination with this waveform data. There is a very good cheat sheet available on nihilogic which I have found very useful. His blog also has many interresting experiments using HTML5 and the canvas tag which should provide some inspiration and insights.

Another good recourse is it the WhatWG HTML5 Draft Specification and in particular the section about the canvas tag.

Example files

To try some of this out immediately download the example project for this tutorial. Such an example iTunes LP is only useful when music is playing and as soon as you start the iTunes LP any music stops. Because of that I have added a search function which you can use to start up music. Just enter a complete album name in the search box and press search & play. This will play a random track from that album and start the visualizer.