Python itertools & functools

These two modules power Python’s functional toolkit:

itertoolsitertools — fast, memory-efficient building blocks for working with iterators.
functoolsfunctools — higher-order functions that act on or return other functions (caching, partial application, reduction).

overview.py

import itertools, functools
 
print(list(itertools.chain([1, 2], [3, 4])))     # [1, 2, 3, 4]
print(functools.reduce(lambda a, b: a + b, [1, 2, 3, 4]))  # 10

overview.py

import itertools, functools
 
print(list(itertools.chain([1, 2], [3, 4])))     # [1, 2, 3, 4]
print(functools.reduce(lambda a, b: a + b, [1, 2, 3, 4]))  # 10

itertools

Iterators produce values lazily — one at a time, without building a whole list in memory. Many itertoolsitertools tools return infinite or large iterators, so wrap them in list(...)list(...) or islice(...)islice(...) to inspect them.

Infinite iterators

Function	Produces
`count(start, step)count(start, step)`	`start, start+step, ...start, start+step, ...` forever.
`cycle(iterable)cycle(iterable)`	Repeats the iterable endlessly.
`repeat(x, n)repeat(x, n)`	`xx`, `nn` times (or forever if `nn` omitted).

infinite.py

from itertools import count, cycle, repeat, islice
 
print(list(islice(count(10, 2), 4)))   # [10, 12, 14, 16]
print(list(islice(cycle("AB"), 5)))    # ['A', 'B', 'A', 'B', 'A']
print(list(repeat("x", 3)))            # ['x', 'x', 'x']

infinite.py

from itertools import count, cycle, repeat, islice
 
print(list(islice(count(10, 2), 4)))   # [10, 12, 14, 16]
print(list(islice(cycle("AB"), 5)))    # ['A', 'B', 'A', 'B', 'A']
print(list(repeat("x", 3)))            # ['x', 'x', 'x']

Combining and slicing iterators

Function	Effect
`chain(a, b, ...)chain(a, b, ...)`	Concatenate iterables end to end.
`islice(it, stop)islice(it, stop)`	Slice an iterator like a list.
`compress(data, sel)compress(data, sel)`	Keep items where the selector is truthy.
`zip_longest(a, b)zip_longest(a, b)`	Like `zipzip` but pads the shorter one.

combining.py

from itertools import chain, islice, compress, zip_longest
 
print(list(chain([1, 2], [3], [4, 5])))            # [1, 2, 3, 4, 5]
print(list(compress("ABCD", [1, 0, 1, 0])))        # ['A', 'C']
print(list(zip_longest([1, 2, 3], ["a"], fillvalue="-")))
# [(1, 'a'), (2, '-'), (3, '-')]

combining.py

from itertools import chain, islice, compress, zip_longest
 
print(list(chain([1, 2], [3], [4, 5])))            # [1, 2, 3, 4, 5]
print(list(compress("ABCD", [1, 0, 1, 0])))        # ['A', 'C']
print(list(zip_longest([1, 2, 3], ["a"], fillvalue="-")))
# [(1, 'a'), (2, '-'), (3, '-')]

Combinatorics

Function	Produces
`product(a, b)product(a, b)`	Cartesian product (nested loops).
`permutations(it, r)permutations(it, r)`	All ordered arrangements of length `rr`.
`combinations(it, r)combinations(it, r)`	All unordered selections of length `rr`.
`combinations_with_replacement(it, r)combinations_with_replacement(it, r)`	Selections allowing repeats.

combinatorics.py

from itertools import product, permutations, combinations
 
print(list(product([1, 2], ["a", "b"])))
# [(1, 'a'), (1, 'b'), (2, 'a'), (2, 'b')]
print(list(permutations([1, 2, 3], 2)))
# [(1, 2), (1, 3), (2, 1), (2, 3), (3, 1), (3, 2)]
print(list(combinations([1, 2, 3], 2)))
# [(1, 2), (1, 3), (2, 3)]

combinatorics.py

from itertools import product, permutations, combinations
 
print(list(product([1, 2], ["a", "b"])))
# [(1, 'a'), (1, 'b'), (2, 'a'), (2, 'b')]
print(list(permutations([1, 2, 3], 2)))
# [(1, 2), (1, 3), (2, 1), (2, 3), (3, 1), (3, 2)]
print(list(combinations([1, 2, 3], 2)))
# [(1, 2), (1, 3), (2, 3)]

Grouping and accumulating

group_accumulate.py

from itertools import groupby, accumulate
 
# groupby groups CONSECUTIVE equal keys -> sort first if needed
data = [("fruit", "apple"), ("fruit", "pear"), ("veg", "kale")]
for key, items in groupby(data, key=lambda pair: pair[0]):
    print(key, [name for _, name in items])
# fruit ['apple', 'pear']
# veg ['kale']
 
# accumulate yields running totals (or any binary function)
print(list(accumulate([1, 2, 3, 4])))            # [1, 3, 6, 10]
print(list(accumulate([1, 2, 3, 4], max)))       # [1, 2, 3, 4]

group_accumulate.py

from itertools import groupby, accumulate
 
# groupby groups CONSECUTIVE equal keys -> sort first if needed
data = [("fruit", "apple"), ("fruit", "pear"), ("veg", "kale")]
for key, items in groupby(data, key=lambda pair: pair[0]):
    print(key, [name for _, name in items])
# fruit ['apple', 'pear']
# veg ['kale']
 
# accumulate yields running totals (or any binary function)
print(list(accumulate([1, 2, 3, 4])))            # [1, 3, 6, 10]
print(list(accumulate([1, 2, 3, 4], max)))       # [1, 2, 3, 4]

functools

reduce

reducereduce folds an iterable into a single value by repeatedly applying a two-argument function.

reduce.py

from functools import reduce
 
print(reduce(lambda a, b: a + b, [1, 2, 3, 4]))      # 10
print(reduce(lambda a, b: a * b, [1, 2, 3, 4]))      # 24
print(reduce(lambda a, b: a if a > b else b, [3, 9, 2]))  # 9 (max)

reduce.py

from functools import reduce
 
print(reduce(lambda a, b: a + b, [1, 2, 3, 4]))      # 10
print(reduce(lambda a, b: a * b, [1, 2, 3, 4]))      # 24
print(reduce(lambda a, b: a if a > b else b, [3, 9, 2]))  # 9 (max)

lru_cache and cache — memoization

@lru_cache@lru_cache stores recent results so repeated calls with the same arguments are instant. Great for expensive or recursive functions.

lru_cache.py

from functools import lru_cache
 
@lru_cache(maxsize=None)   # unbounded cache
def fib(n):
    if n < 2:
        return n
    return fib(n - 1) + fib(n - 2)
 
print(fib(30))             # 832040 — fast, thanks to caching
print(fib.cache_info())    # CacheInfo(hits=..., misses=..., ...)

lru_cache.py

from functools import lru_cache
 
@lru_cache(maxsize=None)   # unbounded cache
def fib(n):
    if n < 2:
        return n
    return fib(n - 1) + fib(n - 2)
 
print(fib(30))             # 832040 — fast, thanks to caching
print(fib.cache_info())    # CacheInfo(hits=..., misses=..., ...)

In Python 3.9+, @functools.cache@functools.cache is a simpler alias for @lru_cache(maxsize=None)@lru_cache(maxsize=None).

partial — pre-fill arguments

partialpartial creates a new function with some arguments already supplied.

partial.py

from functools import partial
 
def power(base, exp):
    return base ** exp
 
square = partial(power, exp=2)
cube = partial(power, exp=3)
print(square(5))   # 25
print(cube(2))     # 8
 
# Common in callbacks: partial(int, base=2) parses binary
to_binary = partial(int, base=2)
print(to_binary("1010"))   # 10

partial.py

from functools import partial
 
def power(base, exp):
    return base ** exp
 
square = partial(power, exp=2)
cube = partial(power, exp=3)
print(square(5))   # 25
print(cube(2))     # 8
 
# Common in callbacks: partial(int, base=2) parses binary
to_binary = partial(int, base=2)
print(to_binary("1010"))   # 10

wraps — preserve metadata in decorators

When you write a decorator, @wraps@wraps copies the original function’s name and docstring onto the wrapper.

wraps.py

from functools import wraps
 
def shout(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs).upper()
    return wrapper
 
@shout
def greet(name):
    "Return a greeting."
    return f"hello {name}"
 
print(greet("ada"))      # HELLO ADA
print(greet.__name__)    # greet  (preserved by @wraps)
print(greet.__doc__)     # Return a greeting.

wraps.py

from functools import wraps
 
def shout(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs).upper()
    return wrapper
 
@shout
def greet(name):
    "Return a greeting."
    return f"hello {name}"
 
print(greet("ada"))      # HELLO ADA
print(greet.__name__)    # greet  (preserved by @wraps)
print(greet.__doc__)     # Return a greeting.

functools tool	Purpose
`reduce(func, it)reduce(func, it)`	Fold an iterable to one value.
`lru_cachelru_cache` / `cachecache`	Memoize function results.
`partial(func, args)partial(func, args)`	Bind some arguments ahead of time.
`wrapswraps`	Keep metadata when writing decorators.
`cmp_to_keycmp_to_key`	Adapt an old-style comparison function for `sortedsorted`.
`total_orderingtotal_ordering`	Fill in missing comparison methods on a class.

Common pitfalls

Iterators are single-use — once consumed, they’re empty. Re-create them to iterate again.
groupbygroupby only groups adjacent equal keys — sort by the key first if groups aren’t contiguous.
Infinite iterators (countcount, cyclecycle) will hang list()list() — slice them with isliceislice.
lru_cachelru_cache requires hashable arguments — you can’t cache calls that take lists or dicts.

Practice Exercises

Exercise 1 – Running totals

Exercise 2 – Multiply a list with reduce

Exercise 3 – Pre-fill an argument with partial

Summary

itertoolsitertools offers lazy building blocks: infinite generators, combiners (chainchain, zip_longestzip_longest), combinatorics (productproduct, permutationspermutations, combinationscombinations), and groupbygroupby/accumulateaccumulate.
Slice infinite iterators with isliceislice; sort before groupbygroupby.
functoolsfunctools adds reducereduce, memoization (lru_cachelru_cache/cachecache), argument binding (partialpartial), and decorator helpers (wrapswraps).
These tools make data pipelines shorter, faster, and clearer.

If this helped you, consider buying me a coffee ☕

Buy me a coffee