Python itertools & functools

These two modules power Python’s functional toolkit:

• itertools — fast, memory-efficient building blocks for working with iterators.
• functools — higher-order functions that act on or return other functions (caching, partial application, reduction).

overview.py

import itertools, functools
 
print(list(itertools.chain([1, 2], [3, 4])))     # [1, 2, 3, 4]
print(functools.reduce(lambda a, b: a + b, [1, 2, 3, 4]))  # 10

itertools

Iterators produce values lazily — one at a time, without building a whole list in memory. Many itertools tools return infinite or large iterators, so wrap them in list(...) or islice(...) to inspect them.

Infinite iterators

Function	Produces
count(start, step)	start, start+step, ... forever.
cycle(iterable)	Repeats the iterable endlessly.
repeat(x, n)	x, n times (or forever if n omitted).

infinite.py

from itertools import count, cycle, repeat, islice
 
print(list(islice(count(10, 2), 4)))   # [10, 12, 14, 16]
print(list(islice(cycle("AB"), 5)))    # ['A', 'B', 'A', 'B', 'A']
print(list(repeat("x", 3)))            # ['x', 'x', 'x']

Combining and slicing iterators

Function	Effect
chain(a, b, ...)	Concatenate iterables end to end.
islice(it, stop)	Slice an iterator like a list.
compress(data, sel)	Keep items where the selector is truthy.
zip_longest(a, b)	Like zip but pads the shorter one.

combining.py

from itertools import chain, islice, compress, zip_longest
 
print(list(chain([1, 2], [3], [4, 5])))            # [1, 2, 3, 4, 5]
print(list(compress("ABCD", [1, 0, 1, 0])))        # ['A', 'C']
print(list(zip_longest([1, 2, 3], ["a"], fillvalue="-")))
# [(1, 'a'), (2, '-'), (3, '-')]

Combinatorics

Function	Produces
product(a, b)	Cartesian product (nested loops).
permutations(it, r)	All ordered arrangements of length r.
combinations(it, r)	All unordered selections of length r.
combinations_with_replacement(it, r)	Selections allowing repeats.

combinatorics.py

from itertools import product, permutations, combinations
 
print(list(product([1, 2], ["a", "b"])))
# [(1, 'a'), (1, 'b'), (2, 'a'), (2, 'b')]
print(list(permutations([1, 2, 3], 2)))
# [(1, 2), (1, 3), (2, 1), (2, 3), (3, 1), (3, 2)]
print(list(combinations([1, 2, 3], 2)))
# [(1, 2), (1, 3), (2, 3)]

Grouping and accumulating

group_accumulate.py

from itertools import groupby, accumulate
 
# groupby groups CONSECUTIVE equal keys -> sort first if needed
data = [("fruit", "apple"), ("fruit", "pear"), ("veg", "kale")]
for key, items in groupby(data, key=lambda pair: pair[0]):
    print(key, [name for _, name in items])
# fruit ['apple', 'pear']
# veg ['kale']
 
# accumulate yields running totals (or any binary function)
print(list(accumulate([1, 2, 3, 4])))            # [1, 3, 6, 10]
print(list(accumulate([1, 2, 3, 4], max)))       # [1, 2, 3, 4]

functools

reduce

reduce folds an iterable into a single value by repeatedly applying a two-argument function.

reduce.py

from functools import reduce
 
print(reduce(lambda a, b: a + b, [1, 2, 3, 4]))      # 10
print(reduce(lambda a, b: a * b, [1, 2, 3, 4]))      # 24
print(reduce(lambda a, b: a if a > b else b, [3, 9, 2]))  # 9 (max)

lru_cache and cache — memoization

@lru_cache stores recent results so repeated calls with the same arguments are instant. Great for expensive or recursive functions.

lru_cache.py

from functools import lru_cache
 
@lru_cache(maxsize=None)   # unbounded cache
def fib(n):
    if n < 2:
        return n
    return fib(n - 1) + fib(n - 2)
 
print(fib(30))             # 832040 — fast, thanks to caching
print(fib.cache_info())    # CacheInfo(hits=..., misses=..., ...)

In Python 3.9+, @functools.cache is a simpler alias for @lru_cache(maxsize=None).

partial — pre-fill arguments

partial creates a new function with some arguments already supplied.

partial.py

from functools import partial
 
def power(base, exp):
    return base ** exp
 
square = partial(power, exp=2)
cube = partial(power, exp=3)
print(square(5))   # 25
print(cube(2))     # 8
 
# Common in callbacks: partial(int, base=2) parses binary
to_binary = partial(int, base=2)
print(to_binary("1010"))   # 10

wraps — preserve metadata in decorators

When you write a decorator, @wraps copies the original function’s name and docstring onto the wrapper.

wraps.py

from functools import wraps
 
def shout(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs).upper()
    return wrapper
 
@shout
def greet(name):
    "Return a greeting."
    return f"hello {name}"
 
print(greet("ada"))      # HELLO ADA
print(greet.__name__)    # greet  (preserved by @wraps)
print(greet.__doc__)     # Return a greeting.

functools tool	Purpose
reduce(func, it)	Fold an iterable to one value.
lru_cache / cache	Memoize function results.
partial(func, *args)	Bind some arguments ahead of time.
wraps	Keep metadata when writing decorators.
cmp_to_key	Adapt an old-style comparison function for sorted.
total_ordering	Fill in missing comparison methods on a class.

sketch One decorator, twenty-five thousand times faster p5.js Pause Restart

Plain recursive fib recomputes the same subproblems over and over: fib(30) makes 2,692,537 calls to evaluate only 31 distinct values. functools.lru_cache stores each result the first time, so every later call is a dictionary lookup. Measured 733.669 ms against 0.02853 ms at n=30. The gap widens with n because the plain version is exponential and the cached one is linear -- this is memoisation, and it is the whole difference between naive recursion and dynamic programming.

Common pitfalls

• Iterators are single-use — once consumed, they’re empty. Re-create them to iterate again.
• groupby only groups adjacent equal keys — sort by the key first if groups aren’t contiguous.
• Infinite iterators (count, cycle) will hang list() — slice them with islice.
• lru_cache requires hashable arguments — you can’t cache calls that take lists or dicts.

Practice Exercises

Exercise 1 – Running totals

pch.exerciseTag pch.exerciseTitle

Exercise 2 – Multiply a list with reduce

pch.exerciseTag pch.exerciseTitle

Exercise 3 – Pre-fill an argument with partial

pch.exerciseTag pch.exerciseTitle

Summary

• itertools offers lazy building blocks: infinite generators, combiners (chain, zip_longest), combinatorics (product, permutations, combinations), and groupby/accumulate.
• Slice infinite iterators with islice; sort before groupby.
• functools adds reduce, memoization (lru_cache/cache), argument binding (partial), and decorator helpers (wraps).
• These tools make data pipelines shorter, faster, and clearer.