Overview

This proposal has progressed to the Draft ECMAScript 6 Specification, which is available for review here: specification_drafts. Any new issues relating to them should be filed as bugs at http://bugs.ecmascript.org. The content on this page is for historic record only and may no longer reflect the current state of the feature described within.

The for-in loop is ECMAScript’s convenient mechanism for doing iteration. However, several factors combine to make it inconvenient for general-purpose iteration.

  • for-in loops iterate over the names of enumerable properties. However properties are not the only type of data set one might want to iterate over.
  • Iterating over an array does not produce each value, but each array index, represented as a string.
  • If an enumerable property is added to Object.prototype, for-in loops visit it.
  • Several aspects of for-in are unspecified in ECMA-262 Edition 5, including the order in which properties are visited.
  • The behavior of for-in is not conveniently customizable. If X is an object representing a binary tree, for example, there is no way to customize X so that for (v in X) instead iterates over all the nodes in the tree. It visits all the enumerable properties of X.

This proposal generalizes the behavior of for-in to obey user-specified iteration protocols via an iteration hook. To preserve compatibility, we add an alternative preposition of, a contextual keyword usable in place of in. This generalization allows the use of for-of loops for custom iterations, providing concise, readable traversals of user-specified data structures and computed value streams.

The same of contextual keyword can be used in place of in in array comprehensions and generator expressions.

Examples

Conveniently iterate over data in Arrays, Sets, and Maps:

for (word of ["one", "two", "three"]) {
    alert(word);
}
 
var s = Set([1, 3, 4, 2, 3, 2, 17, 17, 1, 17]);
for (var v of s) {
    alert(v);
}
 
// Iterating over a Map produces key-value pairs: arrays of length 2.
var m = new Map;
m.set("one", 1);
m.set("two", 2);
for (var [name, value] of m) {
    alert(name + " = " + value);
}

for-of is available in array comprehensions. This makes it easy to convert any iterable object to an Array:

// Make a large Set of strings.
var words = new Set(document.body.textContent.split(/\s+/));
 
// Convert it to an Array and sort the strings.
var arr = [w for w of words];
arr.sort();

Visit the keys of a JSON object:

var obj = JSON.parse(json_data);
for (name of Object.keys(obj))
    alert(name + ": " + obj[name]);

An object is iterable if it has an iterator() method.

for-of simply calls the iterator() method to produce an iterator, then repeatedly calls the iterator’s .next() method to produce values.

function MyCollection() {
    this.elements = [];
}
 
MyCollection.prototype = {
    add: function(x) { this.elements.push(x) },
    iterator: function() {
        return {
            elements: this.elements,
            index: 0,
            next: function() {
                if (this.index >= this.elements.length)
                    throw StopIteration;
                return this.elements[this.index++]
            }
        }
    }
};

A higher-order function that operates on iterables:

function lazyFilter(predicate, data) {
    return {
        sourceIter: data.iterator(),
        iterator: function () { return this; },
        next: function () {
            var item;
            do {
                item = this.sourceIter.next();
            } while (!predicate(item));
            return item;
        }
    };
}

Iterators, for-of, and generators are designed to work together. Another way to write the above function is:

function* lazyFilter(predicate, data) {
    for (var item of data)
        if (predicate(item))
            yield item;
}

And the previous example:

function MyCollection() {
    this.elements = [];
}
 
MyCollection.prototype = {
    add: function(x) { this.elements.push(x) },
    iterator: function*() {
        for (var i = 0; i < this.elements.length; i++)
            yield this.elements[i];
    }
};

Array.prototype.iterator is generic; it will work on any object that has a .length property and elements. So a program can “monkey-patch” existing objects to make them iterable:

// Make DOM NodeLists iterable.
NodeList.prototype.iterator = Array.prototype.iterator;

Of course it is hoped that such hacks will not be necessary very often.

Iteration

Unlike for-in, the for-of mechanism is intended to be extensible with custom iteration behavior.

Iterator objects

In the terminology of this proposal, an iterator is any object of the following form:

{ next: function() -> any }

An object is iterable if it is of the following form:

{ iterator: function() -> iterator }

The .iterator method

The for-of loop syntax supports custom iteration behavior. When for (V of OBJ) STMT executes, first OBJ is evaluated, then an implicit call to OBJ.iterator() occurs. This method is expected to return an iterator. (This is exactly analogous to the implicit call to obj.__iter__() in Python’s for loop, or the implicit call to obj.iterator() in Java’s enhanced for loop.)

Arrays, Maps, Sets, and strings are iterable exactly because there are builtin methods Array.prototype.iterator, Map.prototype.iterator, Set.prototype.iterator, and String.prototype.iterator.

Proxies do not have any special hooks to support iteration. The loop for (V of OBJ) treats OBJ exactly the same whether it is a proxy or not. The proxy handler will receive a get trap for the “iterator” property, just like any other property access.

StopIteration

There is a special “StopIteration” [[Class]] recognized by the for-of loop semantics. When the next method of the iterate trap throws an exception, if the exception is an object whose [[Class]] is “StopIteration”, the loop terminates normally. The exception is not propagated.

There is a standard variable StopIteration that is bound to an object of [[Class]] “StopIteration”.

TODO: StopIteration as a constructor, for compatibility with generators that return values

Iteration semantics

Note: assignments containing a question mark are eliding a simple error propagation, i.e.:

    Let x ?= e

and

    x :?= e

are shorthand for:

    Let x = e
    If IsError(x) Return x

and

    x := e
    If IsError(x) Return x

respectively.

Evaluation of for-of loops

Operation Eval(IterationStatement → for (LHSExpression of Expression) Statement)

Return ForOfLoop(LHSExpression, Expression, Statement, IterationStatement.labels)

Operation Eval(IterationStatement → for (VarDeclarationNoIn of Expression) Statement)

Let varName = Eval(VarDeclarationNoIn)
Let var = IdentifierReference(varName)
Return ForOfLoop(var, Expression, Statement, IterationStatement.labels)

for-of loop bodies

Note: The semantics here have been rewritten to use the .iterator and .next methods. Previously the semantics used an iterate trap, now gone. – Jason Orendorff 6/19/2012.

Operation ForOfLoop(LHS, Expression, Statement, labels)

Let expr ?= Eval(Expression)
Let obj ?= ToObject(expr.value)
Let iteratorMethod ?= obj.[[Get]]("iterator")
Let iterator ?= iteratorMethod.[[Call]](obj, [])
Let V = empty
Repeat
    Let nextMethod ?= iterator.[[Get]]("next")
    Let nextResult = nextMethod.[[Call]](iterator, [])
    If IsError(nextResult)
        If IsObject(nextResult.value) and nextResult.value.[[Class]] = “StopIteration”
            Return (type=normal, value=V, target=empty)
        Return nextResult
    Let lhsRef ?= Eval(LHS)
    Let put ?= PutValue(lhsRef, nextResult.value)
    Let stmt ?= Eval(Statement)
    If stmt.value != empty
        V := stmt.value
    If stmt.type = break
        If stmt.targets ∈ labels
            Return (type=normal, value=V, target=empty)
        Return stmt
    If stmt.type = continue && stmt.target ∉ labels
        Return stmt

TODO: The semantics of a for-of loop where the LHS is bound with let ought to bind the left-hand-side inside the loop instead of outside the loop, so each iteration gets a fresh binding. (See SpiderMonkey bug 449811.)

TODO: The paren-free proposal had for-in automatically let-bound, with no explicit let/var/const keyword. This is an option for for-of as well. This would make it so that iteration only uses binding semantics, never assignment semantics.

Builtin iterators

Array, Map, Set, and String objects each have an iterator method that returns a new iterator each time it is called.

Iterator objects

Iterator.prototype

The initial value of Iterator.prototype is the standard built-in Iterator prototype object, an object with the internal properties { [[NativeBrand]]: NativeIterator, [[Prototype]]: the standard built-in Object prototype object } and the .iterator method below.

This property has the attributes { [[Writable]]: false, [[Enumerable]]: false, [[Configurable]]: false }.

Iterator.prototype.iterator()

  1. Return this.

Array iterators

Array.prototype.iterator works approximately like this:

  Array.prototype.iterator = function* iterator() {
      for (var i = 0; i < (this.length >>> 0); i++)
          yield this[i];
  };

However this is only approximate. Array iterator objects are not generator-iterators.

Array.prototype.iterator()

The following steps are taken:

  1. Let obj = the this value.
  2. Return a new object with the internal properties {[[NativeBrand]]: NativeArrayIterator, [[Prototype]]: the [[ArrayIteratorPrototype]] object, [[Target]]: obj, [[Index]]: 0}.

NOTE This iterator function is intentionally generic.

The [[ArrayIteratorPrototype]] object

The [[ArrayIteratorPrototype]] object is a unique (per global) object. Initially it has the internal properties {[[NativeBrand]]: NativeArrayIterator, [[Prototype]]: the initial value of Iterator.prototype, [[Target]]: undefined, [[Index]]: undefined} and the next method described below.

[[ArrayIteratorPrototype]].next()

The following steps are taken:

  1. Let iter = this.
  2. If iter.[[NativeBrand]] is not ArrayIterator, throw a TypeError.
  3. Let result = ArrayIteratorNext(iter).
  4. If IsError(result), then set iter.[[Index]] = undefined.
  5. Return result.

The abstract operation ArrayIteratorNext(iter) is as follows:

  1. Let i = iter.[[Index]].
  2. If i is undefined, throw StopIteration.
  3. Let target = iter.[[Target]].
  4. Let obj = ToObject(target).
  5. Let length ?= obj.[[Get]](“length”).
  6. Let uintLength ?= ToUint32(length).
  7. If iuintLength, throw StopIteration.
  8. Let value ?= obj.[[Get]](ToString(i).value).
  9. Set iter.[[Index]] = i + 1.
  10. Return value.

NOTE

  • An Array iterator does not skip “array holes”. When it encounters a hole, it consults the target’s prototype chain and yields undefined if no element is found.
  • Array iterators are designed to produce intuitive results when elements are added, modified, or removed to the right of [[Index]]. The iterator continues to produce elements up to the updated .length of the [[Target]]; then it stops.
  • Array iterators are not designed to produce intuitive results when elements are inserted or removed using methods such as Array.prototype.insert, .shift, .unshift, or .splice to the left of [[Index]]. Because these operations do not update the iterator’s internal state, this can cause an iterator to skip elements or produce the same element more than once.
  • If an element has a getter which throws, the exception is propagated, and the iterator is closed. All future calls to .next() will simply throw StopIteration.

String iterators

TODO summary

String.prototype.iterator()

The following steps are taken:

  1. Let s = ToString(this).
  2. Return a new object with the internal properties {[[NativeBrand]]: NativeStringIterator, [[Prototype]]: the [[StringIteratorPrototype]] object, [[Target]]: s, [[Index]]: 0}.

NOTE This causes both primitive strings and String objects to be iterable.

The [[StringIteratorPrototype]] object

The [[StringIteratorPrototype]] object is a unique (per global) object. Initially it has the internal properties {[[NativeBrand]]: NativeStringIterator, [[Prototype]]: the initial value of ''Iterator.prototype'', [[Target]]: the empty string, [[Index]]: undefined} and the next method described below.

[[StringIteratorPrototype]].next()

TODO. Produces the sequence of Unicode characters that make up the string (note: not 16-bit code units).

Standard API

There is a standard “@iter” module which provides the following functionality:

  • isGenerator(x): returns true if x is a generator function, false otherwise.
  • isStopIteration(x): returns true if x is an object whose [[Class]] internal property is “StopIteration”.

as well as some additional convenience functions which behave as follows:

// module "@iter" {
    import create from "@name";
    const call = Function.prototype.call.bind(Function.prototype.call);
    const hasOwn = Object.prototype.hasOwnProperty;
    export function keys(obj) {
        return {
            iterator: function() {
                return (function*() {
                    for (let x in obj) {
                        if (call(hasOwn, obj, x))
                            yield x;
                    }
                })();
            }
        };
    }
    export function values(obj) {
        return {
            iterator: function() {
                return (function*() {
                    for (let x in obj) {
                        if (call(hasOwn, obj, x))
                            yield obj[x];
                    }
                })();
            }
        };
    }
    export function items(obj) {
        return {
            iterator: function() {
                return (function*() {
                    for (let x in obj) {
                        if (call(hasOwn, obj, x))
                            yield [x, obj[x]];
                    }
                })();
            }
        };
    }
    export function allKeys(obj) {
        return {
            iterator: function() {
                return (function*() {
                    for (let x in obj)
                        yield x;
                })();
            }
        };
    }
    export function allValues(obj) {
        return {
            iterator: function() {
                return (function*() {
                    for (let x in obj)
                        yield obj[x];
                })();
            }
        };
    }
    export function allItems(obj) {
        return {
            iterator: function() {
                return (function*() {
                    for (let x in obj)
                        yield [x, obj[x]];
                })();
            }
        };
    }
// }

Rationale

Why throw if iteration is not defined?

  • API evolution: can make an object iterable in v2 without breaking v1 clients
  • clean separation of collections vs reflection

Why the new syntax? Why not extend the for-in semantics?

  • preserve the “LHS is always a string” invariant
  • again, clean separation of collections vs reflection

Resolved Issues

From this thread it’s clear that many people value CoffeeScript’s for own k in o. Should we support for own (k in o) loop head syntax, desugaring to for (k of keys(o))?

Resolved: at the March 2012 TC39 meeting, we agreed not to add for-own-in in favor of iterators. See the meeting notes.

We have long-standing agreement to forbid the =i initializer in for (let x=i in o) and for (let x=i of o) (and paren-free comprehension head variant) forms. Likewise const.

Resolved: fine to remove with TC39 if Mozilla ships it first to prove compatibility claim. See bug 748550.

Brendan Eich 2012/04/24 21:07

Discussion

This proposal have some inconsistencies with previous agreements:

  1. We would use a private name, @iterator, instead of a public name. The main reason for not using an ordinary property name is that it pollutes the property space.
  2. It was previously agreed that the default iterator for strings should iterate over code points, not code units. http://norbertlindenberg.com/2012/05/ecmascript-supplementary-characters/index.html

Erik Arvidsson 2012/06/21 04:24

The public name doesn’t hurt anything, and it reads better.

I’m pleased to hear string iterators will iterate over code points. I just hadn’t heard about that. I’ll update the proposal today.

Jason Orendorff 2012/06/21 13:09

Per https://mail.mozilla.org/pipermail/es-discuss/2012-July/024207.html, we want a unique name for iterator, to support iterable duck-type testing on arbitrary objects.

Brendan Eich 2012/08/31 14:15

 
harmony/iterators.txt · Last modified: 2014/02/20 16:56 by rwaldron
 
Recent changes RSS feed Creative Commons License Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki