arguments
for-of
loopArray.from()
...
)yield*
return()
and throw()
ES6 introduces a new mechanism for traversing data: iteration. Two concepts are central to iteration:
Symbol.iterator
. That method is a factory for iterators.Expressed as interfaces in TypeScript notation, these roles look like this:
The following values are iterable:
Plain objects are not iterable (why is explained in a dedicated section).
Language constructs that access data via iteration:
for-of
loop:
Array.from()
:
...
):
Promise.all()
, Promise.race()
:
yield*
:
The idea of iterability is as follows.
for-of
loops over values and the spread operator (...
) inserts values into Arrays or function calls.It’s not practical for every consumer to support all sources, especially because it should be possible to create new sources (e.g. via libraries). Therefore, ES6 introduces the interface Iterable
. Data consumers use it, data sources implement it:
Given that JavaScript does not have interfaces, Iterable
is more of a convention:
Symbol.iterator
that returns a so-called iterator. The iterator is an object that returns values via its method next()
. We say: it iterates over the items (the content) of the iterable, one per method call.Let’s see what consumption looks like for an Array arr
. First, you create an iterator via the method whose key is Symbol.iterator
:
Then you call the iterator’s method next()
repeatedly to retrieve the items “inside” the Array:
As you can see, next()
returns each item wrapped in an object, as the value of the property value
. The boolean property done
indicates when the end of the sequence of items has been reached.
Iterable
and iterators are part of a so-called protocol (interfaces plus rules for using them) for iteration. A key characteristic of this protocol is that it is sequential: the iterator returns values one at a time. That means that if an iterable data structure is non-linear (such as a tree), iteration will linearize it.
I’ll use the for-of
loop (see Chap. “The for-of
loop”) to iterate over various kinds of iterable data.
Arrays (and Typed Arrays) are iterables over their elements:
Strings are iterable, but they iterate over Unicode code points, each of which may comprise one or two JavaScript characters:
Maps are iterables over their entries. Each entry is encoded as a [key, value] pair, an Array with two elements. The entries are always iterated over deterministically, in the same order in which they were added to the map.
Note that WeakMaps are not iterable.
Sets are iterables over their elements (which are iterated over in the same order in which they were added to the Set).
Note that WeakSets are not iterable.
arguments
Even though the special variable arguments
is more or less obsolete in ECMAScript 6 (due to rest parameters), it is iterable:
Most DOM data structures will eventually be iterable:
Note that implementing this functionality is work in progress. But it is relatively easy to do so, because the symbol Symbol.iterator
can’t clash with existing property keys.
Not all iterable content does have to come from data structures, it could also be computed on the fly. For example, all major ES6 data structures (Arrays, Typed Arrays, Maps, Sets) have three methods that return iterable objects:
entries()
returns an iterable over entries encoded as [key, value] Arrays. For Arrays, the values are the Array elements and the keys are their indices. For Sets, each key and value are the same – the Set element.keys()
returns an iterable over the keys of the entries.values()
returns an iterable over the values of the entries.Let’s see what that looks like. entries()
gives you a nice way to get both Array elements and their indices:
Plain objects (as created by object literals) are not iterable:
Why aren’t objects iterable over properties, by default? The reasoning is as follows. There are two levels at which you can iterate in JavaScript:
Making iteration over properties the default would mean mixing those levels, which would have two disadvantages:
If engines were to implement iterability via a method Object.prototype[Symbol.iterator]()
then there would be an additional caveat: Objects created via Object.create(null)
wouldn’t be iterable, because Object.prototype
is not in their prototype chain.
It is important to remember that iterating over the properties of an object is mainly interesting if you use objects as Maps1. But we only do that in ES5 because we have no better alternative. In ECMAScript 6, we have the built-in data structure Map
.
The proper (and safe) way to iterate over properties is via a tool function. For example, via objectEntries()
, whose implementation is shown later (future ECMAScript versions may have something similar built in):
The following ES6 language constructs make use of the iteration protocol:
for-of
loopArray.from()
...
)Promise.all()
, Promise.race()
yield*
The next sections describe each one of them in detail.
Destructuring via Array patterns works for any iterable:
for-of
loop for-of
is a new loop in ECMAScript 6. It’s basic form looks like this:
For more information, consult Chap. “The for-of
loop”.
Note that the iterability of iterable
is required, otherwise for-of
can’t loop over a value. That means that non-iterable values must be converted to something iterable. For example, via Array.from()
.
Array.from()
Array.from()
converts iterable and Array-like values to Arrays. It is also available for typed Arrays.
For more information on Array.from()
, consult the chapter on Arrays.
...
) The spread operator inserts the values of an iterable into an Array:
That means that it provides you with a compact way to convert any iterable to an Array:
The spread operator also turns an iterable into the arguments of a function, method or constructor call:
The constructor of a Map turns an iterable over [key, value] pairs into a Map:
The constructor of a Set turns an iterable over elements into a Set:
The constructors of WeakMap
and WeakSet
work similarly. Furthermore, Maps and Sets are iterable themselves (WeakMaps and WeakSets aren’t), which means that you can use their constructors to clone them.
Promise.all()
and Promise.race()
accept iterables over Promises:
yield*
yield*
is an operator that is only available inside generators. It yields all items iterated over by an iterable.
The most important use case for yield*
is to recursively call a generator (which produces something iterable).
In this section, I explain in detail how to implement iterables. Note that ES6 generators are usually much more convenient for this task than doing so “manually”.
The iteration protocol looks as follows.
An object becomes iterable (“implements” the interface Iterable
) if it has a method (own or inherited) whose key is Symbol.iterator
. That method must return an iterator, an object that iterates over the items “inside” the iterable via its method next()
.
In TypeScript notation, the interfaces for iterables and iterators look as follows2.
return()
is an optional method that we’ll get to later3. Let’s first implement a dummy iterable to get a feeling for how iteration works.
Let’s check that iterable
is, in fact, iterable:
The code executes three steps, with the counter step
ensuring that everything happens in the right order. First, we return the value 'hello'
, then the value 'world'
and then we indicate that the end of the iteration has been reached. Each item is wrapped in an object with the properties:
value
which holds the actual item anddone
which is a boolean flag that indicates whether the end has been reached, yet.You can omit done
if it is false
and value
if it is undefined
. That is, the switch
statement could be written as follows.
As is explained in the the chapter on generators, there are cases where you want even the last item with done: true
to have a value
. Otherwise, next()
could be simpler and return items directly (without wrapping them in objects). The end of iteration would then be indicated via a special value (e.g., a symbol).
Let’s look at one more implementation of an iterable. The function iterateOver()
returns an iterable over the arguments that are passed to it:
The previous function can be simplified if the iterable and the iterator are the same object:
Even if the original iterable and the iterator are not the same object, it is still occasionally useful if an iterator has the following method (which also makes it an iterable):
All built-in ES6 iterators follow this pattern (via a common prototype, see the chapter on generators). For example, the default iterator for Arrays:
Why is it useful if an iterator is also an iterable? for-of
only works for iterables, not for iterators. Because Array iterators are iterable, you can continue an iteration in another loop:
One use case for continuing an iteration is that you can remove initial items (e.g. a header) before processing the actual content via for-of
.
return()
and throw()
Two iterator methods are optional:
return()
gives an iterator the opportunity to clean up if an iteration ends prematurely.throw()
is about forwarding a method call to a generator that is iterated over via yield*
. It is explained in the chapter on generators.return()
As mentioned before, the optional iterator method return()
is about letting an iterator clean up if it wasn’t iterated over until the end. It closes an iterator. In for-of
loops, premature (or abrupt, in spec language) termination can be caused by:
break
continue
(if you continue an outer loop, continue
acts like a break
)throw
return
In each of these cases, for-of
lets the iterator know that the loop won’t finish. Let’s look at an example, a function readLinesSync
that returns an iterable of text lines in a file and would like to close that file no matter what happens:
Due to return()
, the file will be properly closed in the following loop:
The return()
method must return an object. That is due to how generators handle the return
statement and will be explained in the chapter on generators.
The following constructs close iterators that aren’t completely “drained”:
for-of
yield*
Array.from()
Map()
, Set()
, WeakMap()
, WeakSet()
Promise.all()
, Promise.race()
A later section has more information on closing iterators.
In this section, we look at a few more examples of iterables. Most of these iterables are easier to implement via generators. The chapter on generators shows how.
Tool functions and methods that return iterables are just as important as iterable data structures. The following is a tool function for iterating over the own properties of an object.
Another option is to use an iterator instead of an index to traverse the Array with the property keys:
Combinators4 are functions that combine existing iterables to create new ones.
take(n, iterable)
Let’s start with the combinator function take(n, iterable)
, which returns an iterable over the first n
items of iterable
.
zip(...iterables)
zip
turns n iterables into an iterable of n-tuples (encoded as Arrays of length n).
As you can see, the shortest iterable determines the length of the result:
Some iterable may never be done
.
With an infinite iterable, you must not iterate over “all” of it. For example, by breaking from a for-of
loop:
Or by only accessing the beginning of an infinite iterable:
Or by using a combinator. take()
is one possibility:
The “length” of the iterable returned by zip()
is determined by its shortest input iterable. That means that zip()
and naturalNumbers()
provide you with the means to number iterables of arbitrary (finite) length:
You may be worried about the iteration protocol being slow, because a new object is created for each invocation of next()
. However, memory management for small objects is fast in modern engines and in the long run, engines can optimize iteration so that no intermediate objects need to be allocated. A thread on es-discuss has more information.
In principle, nothing prevents an iterator from reusing the same iteration result object several times – I’d expect most things to work well. However, there will be problems if a client caches iteration results:
If an iterator reuses its iteration result object, iterationResults
will, in general, contain the same object multiple times.
You may be wondering why ECMAScript 6 does not have iterable combinators, tools for working with iterables or for creating iterables. That is because the plans are to proceed in two steps:
Eventually, one such library or pieces from several libraries will be added to the JavaScript standard library.
If you want to get an impression of what such a library could look like, take a look at the standard Python module itertools
.
Yes, iterables are difficult to implement – if you implement them manually. The next chapter will introduce generators that help with this task (among other things).
The iteration protocol comprises the following interfaces (I have omitted throw()
from Iterator
, which is only supported by yield*
and optional there):
Rules for next()
:
x
to produce, next()
returns objects { value: x, done: false }
.next()
should always return an object whose property done
is true
.IteratorResult
The property done
of an iterator result doesn’t have to be true
or false
, truthy or falsy is enough. All built-in language mechanisms let you omit done: false
.
Some iterables produce a new iterator each time they are asked for one. For example, Arrays:
Other iterables return the same iterator each time. For example, generator objects:
Whether an iterable produces a fresh iterators or not matter when you iterate over the same iterable multiple times. For example, via the following function:
With fresh iterators, you can iterate over the same iterable multiple times:
If the same iterator is returned each time, you can’t:
Note that each iterator in the standard library is also an iterable. Its method [Symbol.iterator]()
return this
, meaning that it always returns the same iterator (itself).
The iteration protocol distinguishes two ways of finishing an iterator:
next()
until it returns an object whose property done
is true
.return()
, you tell the iterator that you don’t intend to call next()
, anymore.Rules for calling return()
:
return()
is an optional method, not all iterators have it. Iterators that do have it are called closable.return()
should only be called if an iterator hasn’t be exhausted. For example, for-of
calls return()
whenever it is left “abruptly” (before it is finished). The following operations cause abrupt exits: break
, continue
(with a label of an outer block), return
, throw
.Rules for implementing return()
:
return(x)
should normally produce the object { done: true, value: x }
, but language mechanisms only throw an error (source in spec) if the result isn’t an object.return()
was called, the objects returned by next()
should be done
, too.The following code illustrates that the for-of
loop calls return()
if it is aborted before it receives a done
iterator result. That is, return()
is even called if you abort after receiving the last value. This is subtle and you have to be careful to get it right when you iterate manually or implement iterators.
An iterator is closable if it has a method return()
. Not all iterators are closable. For example, Array iterators are not:
Generator objects are closable by default. For example, the ones returned by the following generator function:
If you invoke return()
on the result of elements()
, iteration is finished:
If an iterator is not closable, you can continue iterating over it after an abrupt exit (such as the one in line A) from a for-of
loop:
Conversely, elements()
returns a closable iterator and the second loop inside twoLoops()
doesn’t have anything to iterate over:
The following class is a generic solution for preventing iterators from being closed. It does so by wrapping the iterator and forwarding all method calls except return()
.
If we use PreventReturn
, the result of the generator elements()
won’t be closed after the abrupt exit in the first loop of twoLoops()
.
There is another way of making generators unclosable: All generator objects produced by the generator function elements()
have the prototype object elements.prototype
. Via elements.prototype
, you can hide the default implementation of return()
(which resides in a prototype of elements.prototype
) as follows:
try-finally
Some generators need to clean up (release allocated resources, close open files, etc.) after iteration over them is finished. Naively, this is how we’d implement it:
In a normal for-of
loop, everything is fine:
However, if you exit the loop after the first yield
, execution seemingly pauses there forever and never reaches the cleanup step:
What actually happens is that, whenever one leaves a for-of
loop early, for-of
sends a return()
to the current iterator. That means that the cleanup step isn’t reached because the generator function returns beforehand.
Thankfully, this is easily fixed, by performing the cleanup in a finally
clause:
Now everything works as desired:
The general pattern for using resources that need to be closed or cleaned up in some manner is therefore:
Note that you must call cleanUp()
when you are going to return a done
iterator result for the first time. You must not do it earlier, because then return()
may still be called. This can be tricky to get right.
If you use iterators, you should close them properly. In generators, you can let for-of
do all the work for you:
If you manage things manually, more work is required:
Even more work is necessary if you don’t use generators:
return()
is called.
try-finally
lets you handle both in a single location.return()
, it should not produce any more iterator results via next()
.for-of
etc.):
return
, if – and only if – you don’t exhaust it. Getting this right can be tricky.