Homepage ▸ Exploring JavaScript (ES2024 Edition)
You can support this book: buy it or donate
(Ad, please don’t block.)

34 Typed Arrays: handling binary data [ES6] (advanced)

34.1 An overview of the API

Much data on the web is text: JSON files, HTML files, CSS files, JavaScript code, etc. JavaScript handles such data well via its built-in strings.

However, before 2011, it did not handle binary data well. The Typed Array Specification 1.0 was introduced on February 8, 2011 and provides tools for working with binary data. With ECMAScript 6, Typed Arrays were added to the core language and gained methods that were previously only available for normal Arrays (.map(), .filter(), etc.).

34.1.1 Use cases for Typed Arrays

The main uses cases for Typed Arrays, are:

34.1.2 The core classes: ArrayBuffer, Typed Arrays, DataView

The Typed Array API stores binary data in instances of ArrayBuffer:

const buf = new ArrayBuffer(4); // length in bytes
  // buf is initialized with zeros

An ArrayBuffer itself is a black box: if we want to access its data, we must wrap it in another object – a view object. Two kinds of view objects are available:

Figure 34.1 shows a class diagram of the API.

Figure 34.1: The classes of the Typed Array API.

34.1.3 SharedArrayBuffer [ES2017]

SharedArrayBuffer is an ArrayBuffer whose memory can be accessed by multiple agents (an agent being the main thread or a web worker) concurrently.

See MDN Web Docs for more information on SharedArrayBuffer and Atomics.

34.2 Using Typed Arrays

Typed Arrays are used much like normal Arrays.

34.2.1 Creating Typed Arrays

The following code shows three different ways of creating the same Typed Array:

// Argument: Typed Array or Array-like object
const ta1 = new Uint8Array([0, 1, 2]);

const ta2 = Uint8Array.of(0, 1, 2);

const ta3 = new Uint8Array(3); // length of Typed Array
ta3[0] = 0;
ta3[1] = 1;
ta3[2] = 2;

assert.deepEqual(ta1, ta2);
assert.deepEqual(ta1, ta3);

34.2.2 The wrapped ArrayBuffer

const typedArray = new Int16Array(2); // 2 elements
assert.equal(typedArray.length, 2);

assert.deepEqual(
  typedArray.buffer, new ArrayBuffer(4)); // 4 bytes

34.2.3 Getting and setting elements

const typedArray = new Int16Array(2);

assert.equal(typedArray[1], 0); // initialized with 0
typedArray[1] = 72;
assert.equal(typedArray[1], 72);

34.2.4 Concatenating Typed Arrays

Typed Arrays don’t have a method .concat(), like normal Arrays do. The workaround is to use their overloaded method .set():

.set(typedArray: TypedArray, offset=0): void
.set(arrayLike: ArrayLike<number>, offset=0): void

It copies the existing typedArray or arrayLike into the receiver, at index offset. TypedArray is an internal abstract superclass of all concrete Typed Array classes (that doesn’t actually have a global name).

The following function uses that method to copy zero or more Typed Arrays (or Array-like objects) into an instance of resultConstructor:

function concatenate(resultConstructor, ...arrays) {
  let totalLength = 0;
  for (const arr of arrays) {
    totalLength += arr.length;
  }
  const result = new resultConstructor(totalLength);
  let offset = 0;
  for (const arr of arrays) {
    result.set(arr, offset);
    offset += arr.length;
  }
  return result;
}
assert.deepEqual(
  concatenate(Uint8Array, Uint8Array.of(1, 2), [3, 4]),
  Uint8Array.of(1, 2, 3, 4));

34.2.5 Typed Arrays vs. normal Arrays

Typed Arrays are much like normal Arrays: they have a .length, elements can be accessed via the bracket operator [], and they have most of the standard Array methods. They differ from normal Arrays in the following ways:

34.3 Using DataViews

This is how DataViews are used:

const dataView = new DataView(new ArrayBuffer(4));
assert.equal(dataView.getInt16(0), 0);
assert.equal(dataView.getUint8(0), 0);
dataView.setUint8(0, 5);

34.4 Element types

ElementTyped ArrayBytesDescriptionGet/Set
Int8Int8Array18-bit signed integernumberES6
Uint8Uint8Array18-bit unsigned intnumberES6
Uint8CUint8ClampedArray18-bit unsigned intnumberES6
(clamped conv.)numberES6
Int16Int16Array216-bit signed intnumberES6
Uint16Uint16Array216-bit unsigned intnumberES6
Int32Int32Array432-bit signed intnumberES6
Uint32Uint32Array432-bit unsigned intnumberES6
BigInt64BigInt64Array864-bit signed intbigintES2020
BigUint64BigUint64Array864-bit unsigned intbigintES2020
Float32Float32Array432-bit floating pointnumberES6
Float64Float64Array864-bit floating pointnumberES6

Table 34.1: Element types supported by the Typed Array API.

Table 34.1 lists the available element types. These types (e.g., Int32) show up in two locations:

The element type Uint8C is special: it is not supported by DataView and only exists to enable Uint8ClampedArray. This Typed Array is used by the canvas element (where it replaces CanvasPixelArray) and should otherwise be avoided. The only difference between Uint8C and Uint8 is how overflow and underflow are handled (as explained in the next subsection).

Typed Arrays and Array Buffers use numbers and bigints to import and export values:

34.4.1 Handling overflow and underflow

Normally, when a value is out of the range of the element type, modulo arithmetic is used to convert it to a value within range. For signed and unsigned integers that means that:

The following function helps illustrate how conversion works:

function setAndGet(typedArray, value) {
  typedArray[0] = value;
  return typedArray[0];
}

Modulo conversion for unsigned 8-bit integers:

const uint8 = new Uint8Array(1);

// Highest value of range
assert.equal(setAndGet(uint8, 255), 255);
// Overflow
assert.equal(setAndGet(uint8, 256), 0);

// Lowest value of range
assert.equal(setAndGet(uint8, 0), 0);
// Underflow
assert.equal(setAndGet(uint8, -1), 255);

Modulo conversion for signed 8-bit integers:

const int8 = new Int8Array(1);

// Highest value of range
assert.equal(setAndGet(int8, 127), 127);
// Overflow
assert.equal(setAndGet(int8, 128), -128);

// Lowest value of range
assert.equal(setAndGet(int8, -128), -128);
// Underflow
assert.equal(setAndGet(int8, -129), 127);

Clamped conversion is different:

const uint8c = new Uint8ClampedArray(1);

// Highest value of range
assert.equal(setAndGet(uint8c, 255), 255);
// Overflow
assert.equal(setAndGet(uint8c, 256), 255);

// Lowest value of range
assert.equal(setAndGet(uint8c, 0), 0);
// Underflow
assert.equal(setAndGet(uint8c, -1), 0);

34.4.2 Endianness

Whenever a type (such as Uint16) is stored as a sequence of multiple bytes, endianness matters:

Endianness tends to be fixed per CPU architecture and consistent across native APIs. Typed Arrays are used to communicate with those APIs, which is why their endianness follows the endianness of the platform and can’t be changed.

On the other hand, the endianness of protocols and binary files varies, but is fixed per format, across platforms. Therefore, we must be able to access data with either endianness. DataViews serve this use case and let us specify endianness when we get or set a value.

Quoting Wikipedia on Endianness:

Other orderings are also possible. Those are generically called middle-endian or mixed-endian.

34.5 Converting to and from Typed Arrays

In this section, «ElementType»Array stands for Int8Array, Uint8Array, etc. ElementType is Int8, Uint8, etc.

34.5.1 The static method «ElementType»Array.from()

This method has the type signature:

.from<S>(
  source: Iterable<S>|ArrayLike<S>,
  mapfn?: S => ElementType, thisArg?: any)
  : «ElementType»Array

.from() converts source into an instance of this (a Typed Array).

For example, normal Arrays are iterable and can be converted with this method:

assert.deepEqual(
  Uint16Array.from([0, 1, 2]),
  Uint16Array.of(0, 1, 2));

Typed Arrays are also iterable:

assert.deepEqual(
  Uint16Array.from(Uint8Array.of(0, 1, 2)),
  Uint16Array.of(0, 1, 2));

source can also be an Array-like object:

assert.deepEqual(
  Uint16Array.from({0:0, 1:1, 2:2, length: 3}),
  Uint16Array.of(0, 1, 2));

The optional mapfn lets us transform the elements of source before they become elements of the result. Why perform the two steps mapping and conversion in one go? Compared to mapping separately via .map(), there are two advantages:

  1. No intermediate Array or Typed Array is needed.
  2. When converting between Typed Arrays with different precisions, less can go wrong.

Read on for an explanation of the second advantage.

34.5.1.1 Pitfall: mapping while converting between Typed Array types

The static method .from() can optionally both map and convert between Typed Array types. Less can go wrong if we use that method.

To see why that is, let us first convert a Typed Array to a Typed Array with a higher precision. If we use .from() to map, the result is automatically correct. Otherwise, we must first convert and then map.

const typedArray = Int8Array.of(127, 126, 125);
assert.deepEqual(
  Int16Array.from(typedArray, x => x * 2),
  Int16Array.of(254, 252, 250));

assert.deepEqual(
  Int16Array.from(typedArray).map(x => x * 2),
  Int16Array.of(254, 252, 250)); // OK
assert.deepEqual(
  Int16Array.from(typedArray.map(x => x * 2)),
  Int16Array.of(-2, -4, -6)); // wrong

If we go from a Typed Array to a Typed Array with a lower precision, mapping via .from() produces the correct result. Otherwise, we must first map and then convert.

assert.deepEqual(
  Int8Array.from(Int16Array.of(254, 252, 250), x => x / 2),
  Int8Array.of(127, 126, 125));

assert.deepEqual(
  Int8Array.from(Int16Array.of(254, 252, 250).map(x => x / 2)),
  Int8Array.of(127, 126, 125)); // OK
assert.deepEqual(
  Int8Array.from(Int16Array.of(254, 252, 250)).map(x => x / 2),
  Int8Array.of(-1, -2, -3)); // wrong

The problem is that if we map via .map(), then input type and output type are the same. In contrast, .from() goes from an arbitrary input type to an output type that we specify via its receiver.

34.5.2 Typed Arrays are iterable

Typed Arrays are iterable. That means that we can use the for-of loop and other iteration-based mechanisms:

const ui8 = Uint8Array.of(0, 1, 2);
for (const byte of ui8) {
  console.log(byte);
}

Output:

0
1
2

ArrayBuffers and DataViews are not iterable.

34.5.3 Converting Typed Arrays to and from normal Arrays

To convert a normal Array to a Typed Array, we pass it to:

For example:

const ta1 = new Uint8Array([0, 1, 2]);
const ta2 = Uint8Array.from([0, 1, 2]);
assert.deepEqual(ta1, ta2);

To convert a Typed Array to a normal Array, we can use Array.from() or spreading (because Typed Arrays are iterable):

assert.deepEqual(
  [...Uint8Array.of(0, 1, 2)], [0, 1, 2]
);
assert.deepEqual(
  Array.from(Uint8Array.of(0, 1, 2)), [0, 1, 2]
);

34.6 Resizing ArrayBuffers [ES2024]

Before ArrayBuffers became resizable, they had fixed sizes. If we wanted one to grow or shrink, we had to allocate a new one and copy the old one over. That costs time and can fragment the address space on 32-bit systems.

34.6.1 New features for ArrayBuffers

These are the changes introduced by resizing:

The options object of the constructor determines whether or not an ArrayBuffer is resizable:

const resizableArrayBuffer = new ArrayBuffer(16, {maxByteLength: 32});
assert.equal(
  resizableArrayBuffer.resizable, true
);

const fixedArrayBuffer = new ArrayBuffer(16);
assert.equal(
  fixedArrayBuffer.resizable, false
);

34.6.2 How Typed Arrays react to changing ArrayBuffer sizes

This is what constructors of Typed Arrays look like:

new «TypedArray»(
  buffer: ArrayBuffer | SharedArrayBuffer,
  byteOffset?: number,
  length?: number
)

If length is undefined then the .length and .byteLength of the Typed Array instance automatically tracks the length of a resizable buffer:

const buf = new ArrayBuffer(2, {maxByteLength: 4});
// `tarr1` starts at offset 0 (`length` is undefined)
const tarr1 = new Uint8Array(buf);
// `tarr2` starts at offset 2 (`length` is undefined)
const tarr2 = new Uint8Array(buf, 2);

assert.equal(
  tarr1.length, 2
);
assert.equal(
  tarr2.length, 0
);

buf.resize(4);

assert.equal(
  tarr1.length, 4
);
assert.equal(
  tarr2.length, 2
);

If an ArrayBuffer is resized then a wrapper with a fixed length can go out of bounds: The wrapper’s range isn’t covered by the ArrayBuffer anymore. That is treated by JavaScript as if the ArrayBuffer were detached:

const buf = new ArrayBuffer(4, {maxByteLength: 4});
const tarr = new Uint8Array(buf, 2, 2);
assert.equal(
  tarr.length, 2
);
buf.resize(3);
// `tarr` is now partially out of bounds
assert.equal(
  tarr.length, 0
);
assert.equal(
  tarr.byteLength, 0
);
assert.equal(
  tarr.byteOffset, 0
);
assert.equal(
  tarr[0], undefined
);
assert.throws(
  () => tarr.at(0),
  /^TypeError: Cannot perform %TypedArray%.prototype.at on a detached ArrayBuffer$/
);

34.6.3 Guidelines given by the ECMAScript specification

The ECMAScript specification gives the following guidelines for working with resizable ArrayBuffers:

34.7 Transferring and detaching ArrayBuffers [ES2024]

34.7.1 Preparation: transferring data and detaching

The web API (not the ECMAScript standard) has long supported structured cloning for safely moving values across realms (globalThis, iframes, web workers, etc.). Some objects can also be transferred: After cloning, the original becomes detached (inaccessible) and ownership switches from the original to the clone. Transfering is usually faster than copying, especially if large amounts of memory are involved. These are the most common classes of transferable objects:

34.7.3 Transferring ArrayBuffers via structuredClone()

The broadly supported structuredClone() also lets us transfer (and therefore detach) ArrayBuffers:

const original = new ArrayBuffer(16);
const clone = structuredClone(original, {transfer: [original]});

assert.equal(
  original.byteLength, 0
);

assert.equal(
  clone.byteLength, 16
);

assert.equal(
  original.detached, true
);
assert.equal(
  clone.detached, false
);

The ArrayBuffer method .transfer() simply gives us a more concise way to detach an ArrayBuffer:

const original = new ArrayBuffer(16);
const transferred = original.transfer();

assert.equal(
  original.detached, true
);
assert.equal(
  transferred.detached, false
);

34.7.4 Transferring an ArrayBuffer within the same agent

Transferring is most often used between two agents (an agent being the main thread or a web worker). However, transferring within the same agent can make sense too: If a function gets a (potentially shared) ArrayBuffer as a parameter, it can transfer it so that no external code can interfere with what it does. Example (taken from the ECMAScript proposal and slightly edited):

async function validateAndWriteSafeAndFast(arrayBuffer) {
  const owned = arrayBuffer.transfer();

  // We have `owned` and no one can access its data via
  // `arrayBuffer` now because the latter is detached:
  assert.equal(
    arrayBuffer.detached, true
  );

  // `await` pauses this function – which gives external
  // code the opportunity to access `arrayBuffer`.
  await validate(owned);
  await fs.writeFile("data.bin", owned);
}

34.7.5 How does detaching an ArrayBuffer affect its wrappers?

34.7.5.1 Typed Arrays with detached ArrayBuffers

Preparation:

> const arrayBuffer = new ArrayBuffer(16);
> const typedArray = new Uint8Array(arrayBuffer);
> arrayBuffer.transfer();

Lengths and offsets are all zero:

> typedArray.length
0
> typedArray.byteLength
0
> typedArray.byteOffset
0

Getting elements returns undefined; setting elements fails silently:

> typedArray[0]
undefined
> typedArray[0] = 128
128

All element-related methods throw exceptions:

> typedArray.at(0)
TypeError: Cannot perform %TypedArray%.prototype.at on a detached ArrayBuffer
34.7.5.2 DataViews with detached ArrayBuffers

All data-related methods of DataViews throw:

> const arrayBuffer = new ArrayBuffer(16);
> const dataView = new DataView(arrayBuffer);
> arrayBuffer.transfer();
> dataView.byteLength
TypeError: Cannot perform get DataView.prototype.byteLength on a detached ArrayBuffer
> dataView.getUint8(0)
TypeError: Cannot perform DataView.prototype.getUint8 on a detached ArrayBuffer
34.7.5.3 We can’t create new wrappers with detached ArrayBuffers
> const arrayBuffer = new ArrayBuffer(16);
> arrayBuffer.transfer();
> new Uint8Array(arrayBuffer)
TypeError: Cannot perform Construct on a detached ArrayBuffer
> new DataView(arrayBuffer)
TypeError: Cannot perform DataView constructor on a detached ArrayBuffer

34.7.6 ArrayBuffer.prototype.transferToFixedLength()

This method rounds out the API: It transfers and converts a resizable ArrayBuffer to one with a fixed length. That may free up memory that was held in preparation for growth.

34.8 Quick references: indices vs. offsets

In preparation for the quick references on ArrayBuffers, Typed Arrays, and DataViews, we need learn the differences between indices and offsets:

Whether a parameter is an index or an offset can only be determined by looking at documentation; there is no simple rule.

34.9 Quick reference: ArrayBuffers

ArrayBuffers store binary data, which is meant to be accessed via Typed Arrays and DataViews.

34.9.1 new ArrayBuffer()

34.9.2 ArrayBuffer.*

34.9.3 ArrayBuffer.prototype.*: getting and slicing

34.9.4 ArrayBuffer.prototype.*: resizing

34.10 Quick reference: Typed Arrays

The properties of the various Typed Array objects are introduced in two steps:

  1. TypedArray: First, we look at the abstract superclass of all Typed Array classes (which was shown in the class diagram at the beginning of this chapter). That superclass is called TypedArray but it does not have a global name in JavaScript:

    > Object.getPrototypeOf(Uint8Array).name
'TypedArray'

  2. «ElementType»Array: The concrete Typed Array classes are called Uint8Array, Int16Array, Float32Array, etc. These are the classes that we use via new, .of, and .from().

34.10.1 TypedArray.*

Both static TypedArray methods are inherited by its subclasses (Uint8Array, etc.). Therefore, we can use these methods via the subclasses, which are concrete and can have direct instances.

34.10.2 TypedArray.prototype.*

Indices accepted by Typed Array methods can be negative (they work like traditional Array methods that way). Offsets must be non-negative. For details, see “Quick references: indices vs. offsets” (§34.8).

34.10.2.1 Properties specific to Typed Arrays

The following properties are specific to Typed Arrays; normal Arrays don’t have them:

34.10.2.2 Array methods

The following methods are basically the same as the methods of normal Arrays (the ECMAScript versions specify when the methods were added to Arrays – Typed Arrays didn’t exist in ECMAScript before ES6):

For details on how these methods work, see “Quick reference: Array” (§33.17).

34.10.3 new «ElementType»Array()

Each Typed Array constructor has a name that follows the pattern «ElementType»Array, where «ElementType» is one of the element types listed in Table 34.1. That means there are 11 constructors for Typed Arrays:

Each constructor has several overloaded versions – it behaves differently depending on how many arguments it receives and what their types are:

34.10.4 «ElementType»Array.*

34.10.5 «ElementType»Array.prototype.*

34.11 Quick reference: DataViews

34.11.1 new DataView()

34.11.2 DataView.prototype.*

In the remainder of this section, «ElementType» refers to either:

These are the properties of DataView.prototype:

Comments
Next: Maps (Map) [ES6]