Skip to content
Python Central Hub

Iterators and Generators in Python

Mastering Iterators and Generators in Python: A Comprehensive Guide

Whenever you write a forfor loop over a list, string, or file, Python is using an iterator behind the scenes. Iterators let you walk through a collection one element at a time without loading everything into memory. Generators are a simple, elegant way to build iterators using the yieldyield keyword. In this guide, we’ll explore the iterator protocol, build custom iterators, and master generators and generator expressions.

What is an Iterable?

An iterable is any object you can loop over — lists, tuples, strings, dictionaries, sets, and files are all iterables. An iterable can be passed to the built-in iter()iter() function to get an iterator.

iterable.py
numbers = [1, 2, 3]
for n in numbers:      # 'numbers' is an iterable
    print(n)
iterable.py
numbers = [1, 2, 3]
for n in numbers:      # 'numbers' is an iterable
    print(n)

What is an Iterator?

An iterator is an object that produces the next value each time you call next()next() on it. An iterator remembers its position. It follows the iterator protocol: it implements two methods, __iter__()__iter__() and __next__()__next__().

iterator.py
numbers = [10, 20, 30]
it = iter(numbers)      # get an iterator from the iterable
print(next(it))         # 10
print(next(it))         # 20
print(next(it))         # 30
print(next(it))         # raises StopIteration
iterator.py
numbers = [10, 20, 30]
it = iter(numbers)      # get an iterator from the iterable
print(next(it))         # 10
print(next(it))         # 20
print(next(it))         # 30
print(next(it))         # raises StopIteration

Output:

command
C:\Users\Your Name> python iterator.py
10
20
30
Traceback (most recent call last):
StopIteration
command
C:\Users\Your Name> python iterator.py
10
20
30
Traceback (most recent call last):
StopIteration

When there are no more values, the iterator raises StopIterationStopIteration. A forfor loop catches this exception automatically and stops looping.

Diagram:

diagram iterator protocol mermaid
How a for loop uses the iterator protocol

Iterable vs. Iterator

ConceptHas __iter__()__iter__()Has __next__()__next__()Example
IterableYesNolistlist, strstr, dictdict
IteratorYesYesresult of iter(list)iter(list)

Building a Custom Iterator

To create your own iterator class, implement __iter__()__iter__() (which returns the iterator object, usually selfself) and __next__()__next__() (which returns the next value or raises StopIterationStopIteration).

custom_iterator.py
class CountUpTo:
    """Iterator that yields numbers from 1 up to a limit."""
    def __init__(self, limit):
        self.limit = limit
        self.current = 0
 
    def __iter__(self):
        return self
 
    def __next__(self):
        if self.current >= self.limit:
            raise StopIteration
        self.current += 1
        return self.current
 
for number in CountUpTo(3):
    print(number)
custom_iterator.py
class CountUpTo:
    """Iterator that yields numbers from 1 up to a limit."""
    def __init__(self, limit):
        self.limit = limit
        self.current = 0
 
    def __iter__(self):
        return self
 
    def __next__(self):
        if self.current >= self.limit:
            raise StopIteration
        self.current += 1
        return self.current
 
for number in CountUpTo(3):
    print(number)

Output:

command
C:\Users\Your Name> python custom_iterator.py
1
2
3
command
C:\Users\Your Name> python custom_iterator.py
1
2
3

What is a Generator?

A generator is a special function that returns an iterator. Instead of returnreturn, it uses the yieldyield keyword. Each time yieldyield runs, the function’s state is frozen and a value is produced; execution resumes from that point on the next next()next() call.

The syntax of a generator function in Python is as follows:

Syntax
def generator_function():
    yield value1
    yield value2
Syntax
def generator_function():
    yield value1
    yield value2

The same CountUpToCountUpTo logic as a generator — far shorter:

generator.py
def count_up_to(limit):
    current = 1
    while current <= limit:
        yield current
        current += 1
 
for number in count_up_to(3):
    print(number)
generator.py
def count_up_to(limit):
    current = 1
    while current <= limit:
        yield current
        current += 1
 
for number in count_up_to(3):
    print(number)

Output:

command
C:\Users\Your Name> python generator.py
1
2
3
command
C:\Users\Your Name> python generator.py
1
2
3

yield vs. return

  • returnreturn sends back a single value and ends the function permanently.
  • yieldyield produces a value and pauses the function, keeping its local state so it can resume later.
yield_pause.py
def demo():
    print("start")
    yield 1
    print("resumed")
    yield 2
    print("done")
 
gen = demo()
print(next(gen))   # prints 'start' then 1
print(next(gen))   # prints 'resumed' then 2
yield_pause.py
def demo():
    print("start")
    yield 1
    print("resumed")
    yield 2
    print("done")
 
gen = demo()
print(next(gen))   # prints 'start' then 1
print(next(gen))   # prints 'resumed' then 2

Output:

command
C:\Users\Your Name> python yield_pause.py
start
1
resumed
2
command
C:\Users\Your Name> python yield_pause.py
start
1
resumed
2

Why Generators Save Memory

A generator produces values lazily — one at a time, only when requested. This means it never builds the whole sequence in memory. Compare creating a list of one million squares versus a generator:

memory.py
import sys
 
# List: stores all million values at once
squares_list = [x * x for x in range(1_000_000)]
print("list bytes:", sys.getsizeof(squares_list))
 
# Generator: stores almost nothing
squares_gen = (x * x for x in range(1_000_000))
print("generator bytes:", sys.getsizeof(squares_gen))
memory.py
import sys
 
# List: stores all million values at once
squares_list = [x * x for x in range(1_000_000)]
print("list bytes:", sys.getsizeof(squares_list))
 
# Generator: stores almost nothing
squares_gen = (x * x for x in range(1_000_000))
print("generator bytes:", sys.getsizeof(squares_gen))

Output:

command
C:\Users\Your Name> python memory.py
list bytes: 8448728
generator bytes: 208
command
C:\Users\Your Name> python memory.py
list bytes: 8448728
generator bytes: 208

Generator Expressions

A generator expression looks like a list comprehension but uses parentheses ()() instead of square brackets [][]. It produces a generator instead of a list.

genexpr.py
# List comprehension - builds a list
squares = [x * x for x in range(5)]
 
# Generator expression - lazy, no list built
lazy_squares = (x * x for x in range(5))
 
print(sum(x * x for x in range(5)))   # pass directly to sum()
genexpr.py
# List comprehension - builds a list
squares = [x * x for x in range(5)]
 
# Generator expression - lazy, no list built
lazy_squares = (x * x for x in range(5))
 
print(sum(x * x for x in range(5)))   # pass directly to sum()

Output:

command
C:\Users\Your Name> python genexpr.py
30
command
C:\Users\Your Name> python genexpr.py
30

Infinite Sequences

Because generators are lazy, they can represent infinite sequences — something impossible with a list.

infinite.py
def naturals():
    n = 1
    while True:         # never stops on its own
        yield n
        n += 1
 
gen = naturals()
for value in gen:
    if value > 5:
        break
    print(value)
infinite.py
def naturals():
    n = 1
    while True:         # never stops on its own
        yield n
        n += 1
 
gen = naturals()
for value in gen:
    if value > 5:
        break
    print(value)

Output:

command
C:\Users\Your Name> python infinite.py
1
2
3
4
5
command
C:\Users\Your Name> python infinite.py
1
2
3
4
5

Visualize it

The magic of a generator is that it’s lazy — it computes each value only when you ask for it with next()next(), and remembers where it left off. Everything past the current point hasn’t been computed yet, which is how a generator can represent a huge or even infinite sequence in almost no memory:

sketch Generators yield one value at a time p5.js
next() produces the next value only when asked. The rest aren't computed yet.

Conclusion

Iterators are objects that produce values one at a time via __iter__()__iter__() and __next__()__next__(), and they power every forfor loop in Python. Generators give you the same behaviour with far less code through the yieldyield keyword, while generator expressions offer a compact, memory-efficient alternative to list comprehensions. Because they produce values lazily, generators can handle huge or even infinite sequences with almost no memory. For more hands-on examples and in-depth tutorials, explore our resources on Python Central Hub!

Try it: Iterators and Generators Exercises

Exercise 1 – Using iter() and next()

Exercise 2 – Write a Generator Function

Exercise 3 – Generator Expression

If this helped you, consider buying me a coffee ☕

Buy me a coffee

Was this page helpful?

Let us know how we did