Description

The widget below demonstrates some basic image processing techniques implemented using the HTML5 <canvas> element. These techniques are presented as a collection of image filters common to most image processing suites including edge detection, sharpening and Gaussian blur. While I will not go into the details at this moment I would like to point out that most of the demonstrated filters are the result of a discrete convolution between the <canvas> elements pixel data matrix and a specially chosen matrix known in image processing as the kernel.

View Demo

Implementation

The implementation of the image filters described above requires access to a canvas elements underlying pixel data. In this section I will provide a brief outline detailing the process involved in accessing and manipulating this data.

HTMLCanvasElement and CanvasRenderingContext2D

The code for this demonstration makes use of four new DOM interfaces introduced in the HTML5 spec. The most important of these interfaces is that of the HTMLCanvasElement which is implemented in all modern browsers as the <canvas> element. As this elements name might suggest it designates a region of fixed width and height onto which bitmap image data and primitives may be drawn and subsequently manipulated. These operations are defined by the the CanvasRenderingContext2D interface. In all browsers which support the <canvas> element a handle to its associated CanvasRenderingContext2D object is returned by the getContext method of the <canvas> element.

• HTML/JavaScript

<canvas id="viewport" height="200" width="200"></canvas>
<script type="text/javascript">
	var canvas = document.getElementById("viewport"); // HTMLCanvasElement
	var context = canvas.getContext('2d'); // CanvasRenderingContext2D
</script>

An image resource may be drawn to the <canvas> element using the drawImage method of the context object.

• JavaScript

var image = new Image();
image.onload = function(){
	context.drawImage(image, 0, 0);
};
image.src = "/img/example.png";

ImageData and Uint8ClampedArray

TheCanvasRenderingContext2D interface provides a number of useful functions to draw to the canvas, however, low level access to the canvas elements underlying pixel data is required in order to implement these filters. An object of type ImageData storing a copy of this data is returned by the CanvasRenderingContext2D method getImageData.

An ImageData object has a data attribute of type Uint8ClampedArray which is a simple one-dimensional array-like object with clamped byte values (0-255) corresponding to the individual RGBa components of the pixel data. In this way a single pixel is stored using four sequential indices of the Uint8ClampedArray object so that the length is equal to four times the pixel count of the corresponding image. Note that in implementations conforming to older versions of the HTML5 specification the ImageData data attribute is of type CanvasPixelArray and unfortunately IE10 also still returns this type.

• JavaScript

var imageData = context.getImageData(0,0, canvas.width, canvas.height); // ImageData
var data = imageData.data; // Uint8ClampedArray
var height = imageData.height;
var width = imageData.width;
console.log(data .toString());
// The previous line prints [object Uint8ClampedArray] in Firefox 18 and Chrome 24
// but prints [object CanvasPixelArray] in IE10

Once the data attribute has been extracted it can be operated on in place. After performing the necessary operations the corresponding ImageData object may be drawn to the canvas using the CanvasRenderingContext2D method putImageData.

• JavaScript

ImageData.prototype.invert = function(){
	var data= this.data;
	var length = data.length;
	for(var idx = 0; idx < length; idx+=4){
		data[idx] = 255 - data[idx]; // red channel
		data[idx + 1] = 255 - data[idx + 1]; // green channel
		data[idx + 2] = 255 - data[idx + 2]; // blue channel
		// data[idx + 3] stores the alpha channel
	}
};
imageData.invert();
context.putImageData(imageData, 0, 0, 0, 0, canvas.width, canvas.height);

Extending Canvas related object prototypes

It may be helpful to extend the prototypes of CanvasRenderingContext2D, ImageData and Uint8ClampedArray. For example Uint8ClampedArray may be extended using the Array.prototype.slice method.

• JavaScript

if(typeof Uint8ClampedArray !== 'undefined'){
	Uint8ClampedArray.prototype.slice = Array.prototype.slice; //Firefox and Chrome
} else if(typeof CanvasPixelArray!== 'undefined') {
	CanvasPixelArray.prototype.slice = Array.prototype.slice; //IE10 and IE9
} else {
	// Deprecated browser
}

Using this assignment the Uint8ClampedArray.prototype.slice method will the return a normal Array. This method could be extended to return the correct type in the latest versions of Firefox (18.0) and Chrome (24.0) using the new Uint8ClampedArray([Array]) constructor but is likely too inefficient to be helpful.