20. Typed Arrays
Table of contents
Please support this book: buy it (PDF, EPUB, MOBI) or donate
(Ad, please don’t block.)

20. Typed Arrays

20.1 Overview

Typed Arrays are an ECMAScript 6 API for handling binary data.

Code example:

const typedArray = new Uint8Array([0,1,2]);
console.log(typedArray.length); // 3
typedArray[0] = 5;
const normalArray = [...typedArray]; // [5,1,2]

// The elements are stored in typedArray.buffer.
// Get a different view on the same data:
const dataView = new DataView(typedArray.buffer);
console.log(dataView.getUint8(0)); // 5

Instances of ArrayBuffer store the binary data to be processed. Two kinds of views are used to access the data:

The following browser APIs support Typed Arrays (details are mentioned in a dedicated section):

20.2 Introduction

Much data one encounters on the web is text: JSON files, HTML files, CSS files, JavaScript code, etc. For handling such data, JavaScript’s built-in string data type works well. However, until a few years ago, JavaScript was ill-equipped to handle binary data. On 8 February 2011, the Typed Array Specification 1.0 standardized facilities for handling binary data. By now, Typed Arrays are well supported by various engines. With ECMAScript 6, they became part of the core language and gained many methods in the process that were previously only available for Arrays (map(), filter(), etc.).

The main uses cases for Typed Arrays are:

Two kinds of objects work together in the Typed Array API:

This is a diagram of the structure of the Typed Array API (notable: all Typed Arrays have a common superclass):

20.2.1 Element types

The following element types are supported by the API:

Element type Bytes Description C type
Int8 1 8-bit signed integer signed char
Uint8 1 8-bit unsigned integer unsigned char
Uint8C 1 8-bit unsigned integer (clamped conversion) unsigned char
Int16 2 16-bit signed integer short
Uint16 2 16-bit unsigned integer unsigned short
Int32 4 32-bit signed integer int
Uint32 4 32-bit unsigned integer unsigned int
Float32 4 32-bit floating point float
Float64 8 64-bit floating point double

The element type Uint8C is special: it is not supported by DataView and only exists to enable Uint8ClampedArray. This Typed Array is used by the canvas element (where it replaces CanvasPixelArray). The only difference between Uint8C and Uint8 is how overflow and underflow are handled (as explained in the next section). It is recommended to avoid the former – quoting Brendan Eich:

Just to be super-clear (and I was around when it was born), Uint8ClampedArray is totally a historical artifact (of the HTML5 canvas element). Avoid unless you really are doing canvas-y things.

20.2.2 Handling overflow and underflow

Normally, when a value is out of the range of the element type, modulo arithmetic is used to convert it to a value within range. For signed and unsigned integers that means that:

Modulo conversion for unsigned 8-bit integers:

> const uint8 = new Uint8Array(1);
> uint8[0] = 255; uint8[0] // highest value within range
255
> uint8[0] = 256; uint8[0] // overflow
0
> uint8[0] = 0; uint8[0] // lowest value within range
0
> uint8[0] = -1; uint8[0] // underflow
255

Modulo conversion for signed 8-bit integers:

> const int8 = new Int8Array(1);
> int8[0] = 127; int8[0] // highest value within range
127
> int8[0] = 128; int8[0] // overflow
-128
> int8[0] = -128; int8[0] // lowest value within range
-128
> int8[0] = -129; int8[0] // underflow
127

Clamped conversion is different:

> const uint8c = new Uint8ClampedArray(1);
> uint8c[0] = 255; uint8c[0] // highest value within range
255
> uint8c[0] = 256; uint8c[0] // overflow
255
> uint8c[0] = 0; uint8c[0] // lowest value within range
0
> uint8c[0] = -1; uint8c[0] // underflow
0

20.2.3 Endianness

Whenever a type (such as Uint16) is stored as multiple bytes, endianness matters:

Endianness tends to be fixed per CPU architecture and consistent across native APIs. Typed Arrays are used to communicate with those APIs, which is why their endianness follows the endianness of the platform and can’t be changed.

On the other hand, the endianness of protocols and binary files varies and is fixed across platforms. Therefore, we must be able to access data with either endianness. DataViews serve this use case and let you specify endianness when you get or set a value.

Quoting Wikipedia on Endianness:

You can use the following function to determine the endianness of a platform.

const BIG_ENDIAN = Symbol('BIG_ENDIAN');
const LITTLE_ENDIAN = Symbol('LITTLE_ENDIAN');
function getPlatformEndianness() {
    const arr32 = Uint32Array.of(0x12345678);
    const arr8 = new Uint8Array(arr32.buffer);
    switch ((arr8[0]*0x1000000) + (arr8[1]*0x10000) + (arr8[2]*0x100) + (arr8\
[3])) {
        case 0x12345678:
            return BIG_ENDIAN;
        case 0x78563412:
            return LITTLE_ENDIAN;
        default:
            throw new Error('Unknown endianness');
    }
}

There are also platforms that arrange words (pairs of bytes) with a different endianness than bytes inside words. That is called mixed endianness. Should you want to support such a platform then it is easy to extend the previous code.

20.2.4 Negative indices

With the bracket operator [ ], you can only use non-negative indices (starting at 0). The methods of ArrayBuffers, Typed Arrays and DataViews work differently: every index can be negative. If it is, it counts backwards from the length. In other words, it is added to the length to produce a normal index. Therefore -1 refers to the last element, -2 to the second-last, etc. Methods of normal Arrays work the same way.

> const ui8 = Uint8Array.of(0, 1, 2);
> ui8.slice(-1)
Uint8Array [ 2 ]

Offsets, on the other hand, must be non-negative. If, for example, you pass -1 to:

DataView.prototype.getInt8(byteOffset)

then you get a RangeError.

20.3 ArrayBuffers

ArrayBuffers store the data, views (Typed Arrays and DataViews) let you read and change it. In order to create a DataView, you need to provide its constructor with an ArrayBuffer. Typed Array constructors can optionally create an ArrayBuffer for you.

20.3.1 ArrayBuffer constructor

The signature of the constructor is:

ArrayBuffer(length : number)

Invoking this constructor via new creates an instance whose capacity is length bytes. Each of those bytes is initially 0.

20.3.2 Static ArrayBuffer methods

20.3.3 ArrayBuffer.prototype properties

20.4 Typed Arrays

The various kinds of Typed Array are only different w.r.t. the type of their elements:

20.4.1 Typed Arrays versus normal Arrays

Typed Arrays are much like normal Arrays: they have a length, elements can be accessed via the bracket operator [ ] and they have all of the standard Array methods. They differ from Arrays in the following ways:

20.4.2 Typed Arrays are iterable

Typed Arrays implement a method whose key is Symbol.iterator and are therefore iterable (consult chapter “Iterables and iterators” for more information). That means that you can use the for-of loop and similar mechanisms in ES6:

const ui8 = Uint8Array.of(0,1,2);
for (const byte of ui8) {
    console.log(byte);
}
// Output:
// 0
// 1
// 2

ArrayBuffers and DataViews are not iterable.

20.4.3 Converting Typed Arrays to and from normal Arrays

To convert a normal Array to a Typed Array, you make it the parameter of a Typed Array constructor. For example:

> const tarr = new Uint8Array([0,1,2]);

The classic way to convert a Typed Array to an Array is to invoke Array.prototype.slice on it. This trick works for all Array-like objects (such as arguments) and Typed Arrays are Array-like.

> Array.prototype.slice.call(tarr)
[ 0, 1, 2 ]

In ES6, you can use the spread operator (...), because Typed Arrays are iterable:

> [...tarr]
[ 0, 1, 2 ]

Another ES6 alternative is Array.from(), which works with either iterables or Array-like objects:

> Array.from(tarr)
[ 0, 1, 2 ]

20.4.4 The Species pattern for Typed Arrays

Some methods create new instances that are similar to this. The species pattern lets you configure what constructor should be used to do so. For example, if you create a subclass MyArray of Array then the default is that map() creates instances of MyArray. If you want it to create instances of Array, you can use the species pattern to make that happen. Details are explained in Sect “The species pattern” in the chapter on classes.

ArrayBuffers use the species pattern in the following locations:

Typed Arrays use the species pattern in the following locations:

DataViews don’t use the species pattern.

20.4.5 The inheritance hierarchy of Typed Arrays

As you could see in the diagram at the beginning of this chapter, all Typed Array classes (Uint8Array etc.) have a common superclass. I’m calling that superclass TypedArray, but it is not directly accessible from JavaScript (the ES6 specification calls it the intrinsic object %TypedArray%). TypedArray.prototype houses all methods of Typed Arrays.

20.4.6 Static TypedArray methods

Both static TypedArray methods are inherited by its subclasses (Uint8Array etc.).

20.4.6.1 TypedArray.of()

This method has the signature:

TypedArray.of(...items)

It creates a new Typed Array that is an instance of this (the class on which of() was invoked). The elements of that instance are the parameters of of().

You can think of of() as a custom literal for Typed Arrays:

> Float32Array.of(0.151, -8, 3.7)
Float32Array [ 0.151, -8, 3.7 ]
20.4.6.2 TypedArray.from()

This method has the signature:

TypedArray<U>.from(source : Iterable<T>, mapfn? : T => U, thisArg?)

It converts the iterable source into an instance of this (a Typed Array).

For example, normal Arrays are iterable and can be converted with this method:

> Uint16Array.from([0, 1, 2])
Uint16Array [ 0, 1, 2 ]

Typed Arrays are iterable, too:

> const ui16 = Uint16Array.from(Uint8Array.of(0, 1, 2));
> ui16 instanceof Uint16Array
true

The optional mapfn lets you transform the elements of source before they become elements of the result. Why perform the two steps mapping and conversion in one go? Compared to performing the first step separately, via source.map(), there are two advantages:

  1. No intermediate Array or Typed Array is needed.
  2. When converting a Typed Array to a Typed Array whose elements have a higher precision, the mapping step can make use of that higher precision.

To illustrate the second advantage, let’s use map() to double the elements of a Typed Array:

> Int8Array.of(127, 126, 125).map(x => 2 * x)
Int8Array [ -2, -4, -6 ]

As you can see, the values overflow and are coerced into the Int8 range of values. If map via from(), you can choose the type of the result so that values don’t overflow:

> Int16Array.from(Int8Array.of(127, 126, 125), x => 2 * x)
Int16Array [ 254, 252, 250 ]

According to Allen Wirfs-Brock, mapping between Typed Arrays was what motivated the mapfn parameter of from().

20.4.7 TypedArray.prototype properties

Indices accepted by Typed Array methods can be negative (they work like traditional Array methods that way). Offsets must be non-negative. For details, see Sect. “Negative indices”.

20.4.7.1 Methods specific to Typed Arrays

The following properties are specific to Typed Arrays, normal Arrays don’t have them:

20.4.7.2 Array methods

The following methods are basically the same as the methods of normal Arrays:

Due to all of these methods being available for Arrays, you can consult the following two sources to find out more about how they work:

Note that while normal Array methods are generic (any Array-like this is OK), the methods listed in this section are not (this must be a Typed Array).

20.4.8 «ElementType»Array constructor

Each Typed Array constructor has a name that follows the pattern «ElementType»Array, where «ElementType» is one of the element types in the table at the beginning. That means that there are 9 constructors for Typed Arrays: Int8Array, Uint8Array, Uint8ClampedArray (element type Uint8C), Int16Array, Uint16Array, Int32Array, Uint32Array, Float32Array, Float64Array.

Each constructor has five overloaded versions – it behaves differently depending on how many arguments it receives and what their types are:

The following code shows three different ways of creating the same Typed Array:

const tarr1 = new Uint8Array([1,2,3]);

const tarr2 = Uint8Array.of(1,2,3);

const tarr3 = new Uint8Array(3);
tarr3[0] = 0;
tarr3[1] = 1;
tarr3[2] = 2;

20.4.9 Static «ElementType»Array properties

20.4.10 «ElementType»Array.prototype properties

20.4.11 Concatenating Typed Arrays

Typed Arrays don’t have a method concat(), like normal Arrays do. The work-around is to use the method

typedArray.set(arrayOrTypedArray, offset=0)

That method copies an existing Typed Array (or normal Array) into typedArray at index offset. Then you only have to make sure that typedArray is big enough to hold all (Typed) Arrays you want to concatenate:

function concatenate(resultConstructor, ...arrays) {
    let totalLength = 0;
    for (const arr of arrays) {
        totalLength += arr.length;
    }
    const result = new resultConstructor(totalLength);
    let offset = 0;
    for (const arr of arrays) {
        result.set(arr, offset);
        offset += arr.length;
    }
    return result;
}
console.log(concatenate(Uint8Array,
    Uint8Array.of(1, 2), Uint8Array.of(3, 4)));
        // Uint8Array [1, 2, 3, 4]

20.5 DataViews

20.5.1 DataView constructor

20.5.2 DataView.prototype properties

20.6 Browser APIs that support Typed Arrays

Typed Arrays have been around for a while, so there are quite a few browser APIs that support them.

20.6.1 File API

The file API lets you access local files. The following code demonstrates how to get the bytes of a submitted local file in an ArrayBuffer.

const fileInput = document.getElementById('fileInput');
const file = fileInput.files[0];
const reader = new FileReader();
reader.readAsArrayBuffer(file);
reader.onload = function () {
    const arrayBuffer = reader.result;
    ···
};

20.6.2 XMLHttpRequest

In newer versions of the XMLHttpRequest API, you can have the results delivered in an ArrayBuffer:

const xhr = new XMLHttpRequest();
xhr.open('GET', someUrl);
xhr.responseType = 'arraybuffer';

xhr.onload = function () {
    const arrayBuffer = xhr.response;
    ···
};

xhr.send();

20.6.3 Fetch API

Similarly to XMLHttpRequest, the Fetch API lets you request resources. But it is based on Promises, which makes it more convenient to use. The following code demonstrates how to download the content pointed to by url as an ArrayBuffer:

fetch(url)
.then(request => request.arrayBuffer())
.then(arrayBuffer => ···);

20.6.4 Canvas

Quoting the HTML5 specification:

The canvas element provides scripts with a resolution-dependent bitmap canvas, which can be used for rendering graphs, game graphics, art, or other visual images on the fly.

The 2D Context of canvas lets you retrieve the bitmap data as an instance of Uint8ClampedArray:

const canvas = document.getElementById('my_canvas');
const context = canvas.getContext('2d');
const imageData = context.getImageData(0, 0, canvas.width, canvas.height);
const uint8ClampedArray = imageData.data;

20.6.5 WebSockets

WebSockets let you send and receive binary data via ArrayBuffers:

const socket = new WebSocket('ws://127.0.0.1:8081');
socket.binaryType = 'arraybuffer';

// Wait until socket is open
socket.addEventListener('open', function (event) {
    // Send binary data
    const typedArray = new Uint8Array(4);
    socket.send(typedArray.buffer);
});

// Receive binary data
socket.addEventListener('message', function (event) {
    const arrayBuffer = event.data;
    ···
});

20.6.6 Other APIs

20.7 Extended example: JPEG SOF0 decoder

The example is a web pages that lets you upload a JPEG file and parses its structure to determine the height and the width of the image and more.

20.7.1 The JPEG file format

A JPEG file is a sequence of segments (typed data). Each segment starts with the following four bytes:

JPEG files are big-endian on all platforms. Therefore, this example demonstrates how important it is that we can specify endianness when using DataViews.

20.7.2 The JavaScript code

The following function processArrayBuffer() is an abridged version of the actual code; I’ve removed a few error checks to reduce clutter. processArrayBuffer() receives an ArrayBuffer with the contents of the submitted JPEG file and iterates over its segments.

// JPEG is big endian
var IS_LITTLE_ENDIAN = false;

function processArrayBuffer(arrayBuffer) {
    try {
        var dv = new DataView(arrayBuffer);
        ···
        var ptr = 2;
        while (true) {
            ···
            var lastPtr = ptr;
            enforceValue(0xFF, dv.getUint8(ptr),
                'Not a marker');
            ptr++;
            var marker = dv.getUint8(ptr);
            ptr++;
            var len = dv.getUint16(ptr, IS_LITTLE_ENDIAN);
            ptr += len;
            logInfo('Marker: '+hex(marker)+' ('+len+' byte(s))');
            ···

            // Did we find what we were looking for?
            if (marker === 0xC0) { // SOF0
                logInfo(decodeSOF0(dv, lastPtr));
                break;
            }
        }
    } catch (e) {
        logError(e.message);
    }
}

This code uses the following helper functions (that are not shown here):

decodeSOF0() parses the segment SOF0:

function decodeSOF0(dv, start) {
    // Example (16x16):
    // FF C0 00 11 08 00 10 00 10 03 01 22 00 02 11 01 03 11 01
    var data = {};
    start += 4; // skip marker 0xFFC0 and segment length 0x0011
    var data = {
        bitsPerColorComponent: dv.getUint8(start), // usually 0x08
        imageHeight: dv.getUint16(start+1, IS_LITTLE_ENDIAN),
        imageWidth: dv.getUint16(start+3, IS_LITTLE_ENDIAN),
        numberOfColorComponents: dv.getUint8(start+5),
    };
    return JSON.stringify(data, null, 4);
}

More information on the structure of JPEG files:

20.8 Availability

Much of the Typed Array API is implemented by all modern JavaScript engines, but several features are new to ECMAScript 6:

It may take a while until these are available everywhere. As usual, kangax’ “ES6 compatibility table” describes the status quo.

Next: 21. Iterables and iterators