Iterators and Generators in Python

Mastering Iterators and Generators in Python: A Comprehensive Guide

Whenever you write a for loop over a list, string, or file, Python is using an iterator behind the scenes. Iterators let you walk through a collection one element at a time without loading everything into memory. Generators are a simple, elegant way to build iterators using the yield keyword. In this guide, we’ll explore the iterator protocol, build custom iterators, and master generators and generator expressions.

What is an Iterable?

An iterable is any object you can loop over — lists, tuples, strings, dictionaries, sets, and files are all iterables. An iterable can be passed to the built-in iter() function to get an iterator.

iterable.py

numbers = [1, 2, 3]
for n in numbers:      # 'numbers' is an iterable
    print(n)

What is an Iterator?

An iterator is an object that produces the next value each time you call next() on it. An iterator remembers its position. It follows the iterator protocol: it implements two methods, __iter__() and __next__().

iterator.py

numbers = [10, 20, 30]
it = iter(numbers)      # get an iterator from the iterable
print(next(it))         # 10
print(next(it))         # 20
print(next(it))         # 30
print(next(it))         # raises StopIteration

Output:

command

C:\Users\Your Name> python iterator.py
10
20
30
Traceback (most recent call last):
StopIteration

When there are no more values, the iterator raises StopIteration. A for loop catches this exception automatically and stops looping.

Diagram:

diagram iterator protocol mermaid

  graph TD
    A[Start for loop] --> B[Call iter on iterable]
    B --> C[Call next on iterator]
    C --> D{StopIteration?}
    D -->|No| E[Run loop body]
    E --> C
    D -->|Yes| F[End loop]

How a for loop uses the iterator protocol

Iterable vs. Iterator

Concept	Has __iter__()	Has __next__()	Example
Iterable	Yes	No	list, str, dict
Iterator	Yes	Yes	result of iter(list)

Note

Every iterator is also an iterable (its __iter__ returns itself), but not every iterable is an iterator. A list is iterable but not an iterator — you cannot call next() on it directly.

Building a Custom Iterator

To create your own iterator class, implement __iter__() (which returns the iterator object, usually self) and __next__() (which returns the next value or raises StopIteration).

custom_iterator.py

class CountUpTo:
    """Iterator that yields numbers from 1 up to a limit."""
    def __init__(self, limit):
        self.limit = limit
        self.current = 0
 
    def __iter__(self):
        return self
 
    def __next__(self):
        if self.current >= self.limit:
            raise StopIteration
        self.current += 1
        return self.current
 
for number in CountUpTo(3):
    print(number)

Output:

command

C:\Users\Your Name> python custom_iterator.py
1
2
3

What is a Generator?

A generator is a special function that returns an iterator. Instead of return, it uses the yield keyword. Each time yield runs, the function’s state is frozen and a value is produced; execution resumes from that point on the next next() call.

The syntax of a generator function in Python is as follows:

Syntax

def generator_function():
    yield value1
    yield value2

The same CountUpTo logic as a generator — far shorter:

generator.py

def count_up_to(limit):
    current = 1
    while current <= limit:
        yield current
        current += 1
 
for number in count_up_to(3):
    print(number)

Output:

command

C:\Users\Your Name> python generator.py
1
2
3

Tip

A function is a generator if it contains at least one yield statement. Calling it does not run the body — it returns a generator object immediately. The body runs only as you iterate.

yield vs. return

• return sends back a single value and ends the function permanently.
• yield produces a value and pauses the function, keeping its local state so it can resume later.

yield_pause.py

def demo():
    print("start")
    yield 1
    print("resumed")
    yield 2
    print("done")
 
gen = demo()
print(next(gen))   # prints 'start' then 1
print(next(gen))   # prints 'resumed' then 2

Output:

command

C:\Users\Your Name> python yield_pause.py
start
1
resumed
2

Why Generators Save Memory

A generator produces values lazily — one at a time, only when requested. This means it never builds the whole sequence in memory. Compare creating a list of one million squares versus a generator:

memory.py

import sys
 
# List: stores all million values at once
squares_list = [x * x for x in range(1_000_000)]
print("list bytes:", sys.getsizeof(squares_list))
 
# Generator: stores almost nothing
squares_gen = (x * x for x in range(1_000_000))
print("generator bytes:", sys.getsizeof(squares_gen))

Output:

command

C:\Users\Your Name> python memory.py
list bytes: 8448728
generator bytes: 208

Generator Expressions

A generator expression looks like a list comprehension but uses parentheses () instead of square brackets []. It produces a generator instead of a list.

genexpr.py

# List comprehension - builds a list
squares = [x * x for x in range(5)]
 
# Generator expression - lazy, no list built
lazy_squares = (x * x for x in range(5))
 
print(sum(x * x for x in range(5)))   # pass directly to sum()

Output:

command

C:\Users\Your Name> python genexpr.py
30

Tip

When a generator expression is the only argument to a function, you can drop the extra parentheses: sum(x*x for x in range(5)).

Infinite Sequences

Because generators are lazy, they can represent infinite sequences — something impossible with a list.

infinite.py

def naturals():
    n = 1
    while True:         # never stops on its own
        yield n
        n += 1
 
gen = naturals()
for value in gen:
    if value > 5:
        break
    print(value)

Output:

command

C:\Users\Your Name> python infinite.py
1
2
3
4
5

Visualize it

The magic of a generator is that it’s lazy — it computes each value only when you ask for it with next(), and remembers where it left off. Everything past the current point hasn’t been computed yet, which is how a generator can represent a huge or even infinite sequence in almost no memory:

sketch Generators yield one value at a time p5.js Pause Restart

next() produces the next value only when asked. The rest aren't computed yet.

Conclusion

Iterators are objects that produce values one at a time via __iter__() and __next__(), and they power every for loop in Python. Generators give you the same behaviour with far less code through the yield keyword, while generator expressions offer a compact, memory-efficient alternative to list comprehensions. Because they produce values lazily, generators can handle huge or even infinite sequences with almost no memory. For more hands-on examples and in-depth tutorials, explore our resources on Python Central Hub!

---

Try it: Iterators and Generators Exercises

Exercise 1 – Using iter() and next()

pch.exerciseTag pch.exerciseTitle

Exercise 2 – Write a Generator Function

pch.exerciseTag pch.exerciseTitle

Exercise 3 – Generator Expression

pch.exerciseTag pch.exerciseTitle