nvie/itertools

A JavaScript port of Python's awesome itertools standard library.

Usage example:

>>> import { izip, cycle } from 'itertools';
>>>
>>> const xs = [1, 2, 3, 4];
>>> const ys = ['hello', 'there'];
>>> for (const [x, y] of izip(xs, cycle(ys))) {
>>>     console.log(x, y);
>>> }
1 'hello'
2 'there'
3 'hello'
4 'there'

About argument order

In Python, many of the itertools take a function as an argument. In the JS port of these we initially kept these orderings the same to stick closely to the Python functions, but in practice, it turns out to be more pragmatic to flip them, so the function gets to be the second param. Example:

In Python:

map(fn, items)

But in JavaScript:

map(items, fn)

The rationale for this flipping of argument order is because in practice, the function bodies can span multiple lines, in which case the following block will remain aesthetically pleasing:

import { map } from "itertools";

const numbers = [1, 2, 3];
const squares = map(numbers, (n) => {
  //
  // Do something wild with these numbers here
  //
  // ...
  return n * n;
});

API

The itertools package consists of a few building blocks:

• Ports of builtins
• Ports of itertools
• Ports of more-itertools
• Additions

Ports of builtins

• every
• some
• contains
• enumerate
• filter
• iter
• map
• max
• min
• range
• reduce
• sorted
• xrange
• sum
• zip
• zip3

# every(iterable: Iterable<T>, keyFn?: Predicate<T>): boolean <>

Returns true when every of the items in iterable are truthy. An optional key function can be used to define what truthiness means for this specific collection.

Examples:

every([]); // => true
every([0]); // => false
every([0, 1, 2]); // => false
every([1, 2, 3]); // => true

Examples with using a key function:

every([2, 4, 6], (n) => n % 2 === 0); // => true
every([2, 4, 5], (n) => n % 2 === 0); // => false

# some(iterable: Iterable<T>, keyFn?: Predicate<T>): boolean <>

Returns true when some of the items in iterable are truthy. An optional key function can be used to define what truthiness means for this specific collection.

Examples:

some([]); // => false
some([0]); // => false
some([0, 1, null, undefined]); // => true

Examples with using a key function:

some([1, 4, 5], (n) => n % 2 === 0); // => true
some([{ name: "Bob" }, { name: "Alice" }], (person) => person.name.startsWith("C")); // => false

# contains(haystack: Iterable<T>, needle: T): boolean <>

Returns true when some of the items in the iterable are equal to the target object.

Examples:

contains([], "whatever"); // => false
contains([3], 42); // => false
contains([3], 3); // => true
contains([0, 1, 2], 2); // => true

# enumerate(iterable: Iterable<T>, start: number = 0): Iterable<[number, T]> <>

Returns an iterable of enumeration pairs. Iterable must be a sequence, an iterator, or some other object which supports iteration. The elements produced by returns a tuple containing a counter value (starting from 0 by default) and the values obtained from iterating over given iterable.

Example:

import { enumerate } from "itertools";

console.log([...enumerate(["hello", "world"])]);
// [0, 'hello'], [1, 'world']]

# filter(iterable: Iterable<T>, predicate: Predicate<T>): T[] <>

Eager version of ifilter.

# iter(iterable: Iterable<T>): Iterator<T> <>

Returns an iterator object for the given iterable. This can be used to manually get an iterator for any iterable datastructure. The purpose and main use case of this function is to get a single iterator (a thing with state, think of it as a "cursor") which can only be consumed once.

# map(iterable: Iterable<T>, mapper: (item: T) => V): V[] <>

Eager version of imap.

# max(iterable: Iterable<T>, keyFn?: (item: T) => number): T | undefined <>

Return the largest item in an iterable. Only works for numbers, as ordering is pretty poorly defined on any other data type in JS. The optional keyFn argument specifies a one-argument ordering function like that used for sorted.

If the iterable is empty, undefined is returned.

If multiple items are maximal, the function returns either one of them, but which one is not defined.

# min(iterable: Iterable<T>, keyFn?: (item: T) => number): T | undefined <>

Return the smallest item in an iterable. Only works for numbers, as ordering is pretty poorly defined on any other data type in JS. The optional keyFn argument specifies a one-argument ordering function like that used for sorted.

If the iterable is empty, undefined is returned.

If multiple items are minimal, the function returns either one of them, but which one is not defined.

# range(stop: number): Iterable<number> <>
# range(start: number, stop: number, step: number = 1): Iterable<number> <>

Returns an iterator producing all the numbers in the given range one by one, starting from start (default 0), as long as i < stop, in increments of step (default 1).

range(a) is a convenient shorthand for range(0, a).

Various valid invocations:

range(5)           // 0, 1, 2, 3, 4
range(0, 5)        // 0, 1, 2, 3, 4
range(0, 5, 2)     // 0, 2, 4
range(5, 0, -1)    // 5, 4, 3, 2, 1
range(5, 0)        // (empty)
range(-3)          // (empty)

For a positive step, the iterator will keep producing values n as long as the stop condition n < stop is satisfied.

For a negative step, the iterator will keep producing values n as long as the stop condition n > stop is satisfied.

The produced range will be empty if the first value to produce already does not meet the value constraint.

# xrange(stop: number): number[] <>
# xrange(start: number, stop: number, step: number = 1): number[] <>

Returns an array with all the numbers in the given range, starting from start (default 0), as long as i < stop, in increments of step (default 1).

xrange(5) is a convenient shorthand for Array.from(range(5)).

Various valid invocations:

xrange(5)           // [0, 1, 2, 3, 4]
xrange(2, 5)        // [2, 3, 4]
xrange(0, 5, 2)     // [0, 2, 4]
xrange(5, 0, -1)    // [5, 4, 3, 2, 1]
xrange(5, 0)        // []

For a positive step, the iterator will keep producing values n as long as the stop condition n < stop is satisfied.

For a negative step, the iterator will keep producing values n as long as the stop condition n > stop is satisfied.

The produced range will be empty if the first value to produce already does not meet the value constraint.

Don't use this on large or infinite ranges, as it will allocate a large array in memory.

# reduce(iterable: Iterable<T>, reducer: (O, T, number) => O, start: O): O <>
# reduce(iterable: Iterable<T>, reducer: (T, T, number) => T): T | undefined <>

Apply function of two arguments cumulatively to the items of sequence, from left to right, so as to reduce the sequence to a single value. For example:

reduce([1, 2, 3, 4, 5], (total, x) => total + x, 0);

calculates

(((((0+1)+2)+3)+4)+5)

The left argument, total, is the accumulated value and the right argument, x, is the update value from the sequence.

Without an explicit initializer arg:

reduce([1, 2, 3, 4, 5], (total, x) => total + x);

it calculates

((((1+2)+3)+4)+5)

# sorted(iterable: Iterable<T>, keyFn?: (item: T) => Primitive, reverse?: boolean): T[] <>

Return a new sorted list from the items in iterable.

Has two optional arguments:

• keyFn specifies a function of one argument providing a primitive identity for each element in the iterable. that will be used to compare. The default value is to use a default identity function that is only defined for primitive types.

• reverse is a boolean value. If true, then the list elements are sorted as if each comparison were reversed.

# sum(iterable: Iterable<number>): number <>

Sums the items of an iterable from left to right and returns the total. The sum will defaults to 0 if the iterable is empty.

# zip(xs: Iterable<T1>, ys: Iterable<T2>): [T1, T2][] <>
# zip3(xs: Iterable<T1>, ys: Iterable<T2>, zs: Iterable<T3>): [T1, T2, T3][] <>

Eager version of izip / izip3.

Ports of itertools

• chain
• compress
• count
• cycle
• dropwhile
• groupBy
• icompress
• ifilter
• igroupby
• imap
• indexBy
• islice
• izip
• izip3
• izipLongest
• izipMany
• permutations
• repeat
• takewhile
• zipLongest
• zipMany

# chain(...iterables: Iterable<T>[]): Iterable<T> <>

Returns an iterator that returns elements from the first iterable until it is exhausted, then proceeds to the next iterable, until all of the iterables are exhausted. Used for treating consecutive sequences as a single sequence.

# compress(iterable: Iterable<T>, selectors: Iterable<boolean>): T[] <>

Eager version of icompress.

# count(start: number, step: number): Iterable<number> <>

Returns an iterator that counts up values starting with number start (default 0), incrementing by step. To decrement, use a negative step number.

# cycle(iterable: Iterable<T>): Iterable<T> <>

Returns an iterator producing elements from the iterable and saving a copy of each. When the iterable is exhausted, return elements from the saved copy. Repeats indefinitely.

# dropwhile(iterable: Iterable<T>, predicate: (item: T) => boolean): Iterable<T> <>

Returns an iterator that drops elements from the iterable as long as the predicate is true; afterwards, returns every remaining element. Note: the iterator does not produce any output until the predicate first becomes false.

# groupBy(iterable: Iterable<T>, keyFn: (item: T) => K): Record<K, T[]> <>

Groups elements of the iterable into a record based on the key function. Each key maps to an array of all elements that share the same key.

const users = [
  { name: "Alice", department: "Engineering" },
  { name: "Bob", department: "Sales" },
  { name: "Charlie", department: "Engineering" },
];

groupBy(users, (user) => user.department);
// {
//   'Engineering': [
//     { name: 'Alice', department: 'Engineering' },
//     { name: 'Charlie', department: 'Engineering' }
//   ],
//   'Sales': [
//     { name: 'Bob', department: 'Sales' }
//   ]
// }

# icompress(iterable: Iterable<T>, selectors: Iterable<boolean>): Iterable<T> <>

Returns an iterator that filters elements from data returning only those that have a corresponding element in selectors that evaluates to true. Stops when either the data or selectors iterables has been exhausted.

# ifilter(iterable: Iterable<T>, predicate: Predicate<T>): Iterable<T> <>

Returns an iterator that filters elements from iterable returning only those for which the predicate is true.

# igroupby(iterable: Iterable<T>, keyFn: (item: T) => Primitive): Iterable<[Primitive, Iterable<T>]> <>

Make an Iterable that returns consecutive keys and groups from the iterable. The key is a function computing a key value for each element. If not specified, key defaults to an identity function and returns the element unchanged. Generally, the iterable needs to already be sorted on the same key function.

The operation of igroupby() is similar to the uniq filter in Unix. It generates a break or new group every time the value of the key function changes (which is why it is usually necessary to have sorted the data using the same key function). That behavior differs from SQL’s GROUP BY which aggregates common elements regardless of their input order.

The returned group is itself an iterator that shares the underlying iterable with igroupby(). Because the source is shared, when the igroupby() object is advanced, the previous group is no longer visible. So, if that data is needed later, it should be stored as an array.

# imap(iterable: Iterable<T>, mapper: (item: T) => V): Iterable<V> <>

Returns an iterator that computes the given mapper function using arguments from each of the iterables.

# indexBy(iterable: Iterable<T>, keyFn: (item: T) => K): Record<K, T> <>

Creates an index (record) from the iterable where each key maps to the last element that produced that key. If multiple elements produce the same key, only the last one is kept.

const users = [
  { id: 1, name: "Alice" },
  { id: 2, name: "Bob" },
  { id: 3, name: "Charlie" },
];

indexBy(users, (user) => user.id);
// {
//   1: { id: 1, name: 'Alice' },
//   2: { id: 2, name: 'Bob' },
//   3: { id: 3, name: 'Charlie' }
// }

# islice(iterable: Iterable<T>[start: number], stop: number[, step: number]): Iterable<T> <>

Returns an iterator that returns selected elements from the iterable. If start is non-zero, then elements from the iterable are skipped until start is reached. Then, elements are returned by making steps of step (defaults to 1). If set to higher than 1, items will be skipped. If stop is provided, then iteration continues until the iterator reached that index, otherwise, the iterable will be fully exhausted. islice() does not support negative values for start, stop, or step.

# izip(xs: Iterable<T1>, ys: Iterable<T2>): Iterable<[T1, T2]> <>
# izip3(xs: Iterable<T1>, ys: Iterable<T2>, zs: Iterable<T3>): Iterable<[T1, T2, T3]> <>

Returns an iterator that aggregates elements from each of the iterables. Used for lock-step iteration over several iterables at a time. When iterating over two iterables, use izip2. When iterating over three iterables, use izip3, etc. izip is an alias for izip2.

# izipLongest(xs: Iterable<T1>, ys: Iterable<T2>, filler?: D): Iterable<[T1 | D, T2 | D]> <>
# izipLongest3(xs: Iterable<T1>, ys: Iterable<T2>, zs: Iterable<T3>, filler?: D): Iterable<[T1 | D, T2 | D, T3 | D]> <>

Returns an iterator that aggregates elements from each of the iterables. If the iterables are of uneven length, missing values are filled-in with fillvalue. Iteration continues until the longest iterable is exhausted.

# izipMany(...iters: Iterable<T>[]): Iterable<T[]> <>

Like the other izips (izip, izip3, etc), but generalized to take an unlimited amount of input iterables. Think izip(*iterables) in Python.

# permutations(iterable: Iterable<T>, r: number = undefined): Iterable<T[]> <>

Return successive r-length permutations of elements in the iterable.

If r is not specified, then r defaults to the length of the iterable and all possible full-length permutations are generated.

Permutations are emitted in lexicographic sort order. So, if the input iterable is sorted, the permutation tuples will be produced in sorted order.

Elements are treated as unique based on their position, not on their value. So if the input elements are unique, there will be no repeat values in each permutation.

# repeat(thing: T, times: number = undefined): Iterable<T> <>

Returns an iterator that produces values over and over again. Runs indefinitely unless the times argument is specified.

# takewhile(iterable: Iterable<T>, predicate: (item: T) => boolean): Iterable<T> <>

Returns an iterator that produces elements from the iterable as long as the predicate is true.

# zipLongest(xs: Iterable<T1>, ys: Iterable<T2>, filler?: D): [T1 | D, T2 | D][] <>
# zipLongest3(xs: Iterable<T1>, ys: Iterable<T2>, zs: Iterable<T3>, filler?: D): [T1 | D, T2 | D, T3 | D][] <>

Eager version of izipLongest and friends.

# zipMany(...iters: Iterable<T>[]): T[][] <>

Eager version of izipMany.

Ports of more-itertools

• chunked
• flatten
• intersperse
• itake
• pairwise
• partition
• roundrobin
• heads
• take
• uniqueEverseen
• uniqueJustseen
• dupes

# chunked(iterable: Iterable<T>, size: number): Iterable<T[]> <>

Break iterable into lists of length size:

>>> [...chunked([1, 2, 3, 4, 5, 6], 3)]
[[1, 2, 3], [4, 5, 6]]

If the length of iterable is not evenly divisible by size, the last returned list will be shorter:

>>> [...chunked([1, 2, 3, 4, 5, 6, 7, 8], 3)]
[[1, 2, 3], [4, 5, 6], [7, 8]]

# flatten(iterableOfIterables: Iterable<Iterable<T>>): Iterable<T> <>

Return an iterator flattening one level of nesting in a list of lists:

>>> [...flatten([[0, 1], [2, 3]])]
[0, 1, 2, 3]

# intersperse(value: T, iterable: Iterable<T>): Iterable<T> <>

Intersperse filler element value among the items in iterable.

>>> [...intersperse(-1, range(1, 5))]
[1, -1, 2, -1, 3, -1, 4]

# itake(n: number, iterable: Iterable<T>): Iterable<T> <>

Returns an iterable containing only the first n elements of the given iterable.

# pairwise(iterable: Iterable<T>): Iterable<[T, T]> <>

Returns an iterator of paired items, overlapping, from the original. When the input iterable has a finite number of items n, the outputted iterable will have n - 1 items.

>>> pairwise([8, 2, 0, 7])
[(8, 2), (2, 0), (0, 7)]

# partition(iterable: Iterable<T>, predicate: Predicate<T>): [T[], T[]] <>

Returns a 2-tuple of arrays. Splits the elements in the input iterable into either of the two arrays. Will fully exhaust the input iterable. The first array contains all items that match the predicate, the second the rest:

>>> const isOdd = x => x % 2 !== 0;
>>> const iterable = range(10);
>>> const [odds, evens] = partition(iterable, isOdd);
>>> odds
[1, 3, 5, 7, 9]
>>> evens
[0, 2, 4, 6, 8]

# roundrobin(...iterables: Iterable<T>[]): Iterable<T> <>

Yields the next item from each iterable in turn, alternating between them. Continues until all items are exhausted.

>>> [...roundrobin([1, 2, 3], [4], [5, 6, 7, 8])]
[1, 4, 5, 2, 6, 3, 7, 8]

# heads(...iterables: Iterable<T>[]): Iterable<T[]> <>

Like roundrobin(), but will group the output per "round".

>>> [...heads([1, 2, 3], [4], [5, 6, 7, 8])]
[[1, 4, 5], [2, 6], [3, 7], [8]]

# take(n: number, iterable: Iterable<T>): T[] <>

Eager version of itake.

# uniqueEverseen(iterable: Iterable<T>, keyFn?: (item: T) => Primitive): Iterable<T> <>

Yield unique elements, preserving order.

>>> [...uniqueEverseen('AAAABBBCCDAABBB')]
['A', 'B', 'C', 'D']
>>> [...uniqueEverseen('AbBCcAB', s => s.toLowerCase())]
['A', 'b', 'C']

# uniqueJustseen(iterable: Iterable<T>, keyFn?: (item: T) => Primitive): Iterable<T> <>

Yields elements in order, ignoring serial duplicates.

>>> [...uniqueJustseen('AAAABBBCCDAABBB')]
['A', 'B', 'C', 'D', 'A', 'B']
>>> [...uniqueJustseen('AbBCcAB', s => s.toLowerCase())]
['A', 'b', 'C', 'A', 'B']

# dupes(iterable: Iterable<T>, keyFn?: (item: T) => Primitive): Iterable<T[]> <>

Yield only elements from the input that occur more than once. Needs to consume the entire input before being able to produce the first result.

>>> [...dupes('AAAABCDEEEFABG')]
[['A', 'A', 'A', 'A', 'A'], ['E', 'E', 'E'], ['B', 'B']]
>>> [...dupes('AbBCcAB', s => s.toLowerCase())]
[['b', 'B', 'B'], ['C', 'c'], ['A', 'A']]

Additions

• chunkedByCost
• compact
• compactObject
• find
• first
• flatmap
• icompact

# chunkedByCost(items: Iterable<T>, costOf: (item: T) => number, maxCost: number, minCost?: number | ((chunkIndex: number) => number)): Iterable<T[]> <>

Group items from a stream into chunks, where each chunk's total cost stays under maxCost (hard cap) and optionally reaches minCost (soft target).

maxCost is a hard cap: a chunk is flushed before adding an item that would push the running cost past maxCost.

minCost is an optional soft target. When provided, a chunk is flushed as soon as the running cost reaches it. It may be:

• a number: same target for every chunk
• a function (chunkIndex: number) => number: per-chunk target, indexed by the 0-based output chunk number. Useful for ramp-up schedules.

When minCost is omitted, chunks grow as large as maxCost allows.

>>> // Pack rows into batches of <= 10 bytes
>>> const rows = ["aaaa", "bbbb", "cc", "dddddd", "ee"];
>>> [...chunkedByCost(rows, (s) => s.length, 10)]
[["aaaa", "bbbb", "cc"], ["dddddd", "ee"]]

With a constant minCost soft target, the chunker flushes early to keep chunks roughly that size — useful when you want predictable chunk sizes without locking them to a fixed item count:

>>> [...chunkedByCost(rows, (s) => s.length, 100, 6)]
[["aaaa", "bbbb"], ["cc", "dddddd"], ["ee"]]

With a scheduled minCost, early chunks can target smaller sizes than later ones — useful for fast-start patterns (e.g. emit a few small chunks first to clear a slow outgoing queue, then settle into larger ones):

>>> const items = Array.from({ length: 30 }, () => 1);
>>> [...chunkedByCost(items, () => 1, 100, (i) => Math.min(8, 1 << i))]
[[1], [1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1], ...]

If a single item's cost alone is >= maxCost, it is still emitted as its own chunk. Only in this edge case can an output chunk exceed the hard cap.

# compact(iterable: Iterable<T | null | undefined>): T[] <>

Eager version of icompact.

# compactObject(obj: Record<K, V | null | undefined>): Record<K, V> <>

Removes all "nullish" values from the given object. Returns a new object.

>>> compactObject({ a: 1, b: undefined, c: 0, d: null })
{ a: 1, c: 0, d: null }

# find(iterable: Iterable<T>, keyFn?: Predicate<T>): T | undefined <>

Returns the first item in the iterable for which the predicate holds, if any. If no such item exists, undefined is returned. If no default predicate is given, the first value from the iterable is returned.

# first(iterable: Iterable<T>, keyFn?: Predicate<T>): T | undefined <>

Almost the same as find(), except when no explicit predicate function is given. find() will always return the first value in the iterable, whereas first() will return the first non-undefined value in the iterable.

Prefer using find(), as its behavior is more intuitive and predictable.

# flatmap(iterable: Iterable<T>, mapper: (item: T) => Iterable<S>): Iterable<S> <>

Returns 0 or more values for every value in the given iterable. Technically, it's just calling map(), followed by flatten(), but it's a very useful operation if you want to map over a structure, but not have a 1:1 input-output mapping. Instead, if you want to potentially return 0 or more values per input element, use flatmap():

For example, to return all numbers n in the input iterable n times:

>>> const repeatN = n => repeat(n, n);
>>> [...flatmap([0, 1, 2, 3, 4], repeatN)]
[1, 2, 2, 3, 3, 3, 4, 4, 4, 4]  // note: no 0

# icompact(iterable: Iterable<T | null | undefined>): Iterable<T> <>

Returns an iterable, filtering out any "nullish" values from the input iterable.

>>> compact([1, 2, undefined, 3])
[1, 2, 3]