Python Generators

A generator function produces values one at a time and remembers where it stopped. That makes it possible to work with long or infinite sequences without holding every value in memory. The key is yield, which pauses the function instead of returning from it permanently. You will also delegate with yield from, observe laziness, and run cleanup with close().

Yield one value at a time

When the function reaches yield, it hands one value back to the caller and pauses. The next request resumes after that line. A loop can consume the whole sequence without knowing how many values will be produced.

def countdown(n):
    while n > 0:
        yield n
        n -= 1

for value in countdown(3):
    print(value)

A paused function

Pull values with next

Calling a generator function does not run the body immediately. It returns a generator object. Each call to next asks for one value. When there are no more values, next raises StopIteration, which a for loop handles for you.

def numbers():
    yield 1
    yield 2

gen = numbers()
print(next(gen))
print(next(gen))

Demand-driven values

Use a generator expression

A generator expression uses parentheses instead of the square brackets of a list comprehension. It produces values lazily. Wrap it in list() when you need to inspect or reuse the complete result.

squares = (n * n for n in range(4))
print(list(squares))

Lazy comprehension

yield from delegates a stream

yield from hands control to another iterable and forwards each value. It replaces a nested for/yield loop and makes composition explicit. The outer generator resumes after the inner one is exhausted.

def drinks():
    yield "latte"
    yield "tea"

def menu():
    yield "menu"
    yield from drinks()
    yield "done"

print(list(menu()))

Forward values from a sub-generator

A generator pipeline stays lazy

Creating a generator does not run its body. Values move only when a consumer asks for the next item, so a pipeline can stop early. Side effects such as logging happen at consumption time, not at pipeline construction.

def source():
    print("source start")
    for n in [1, 2, 3]:
        print("yield", n)
        yield n

pipeline = (n * 10 for n in source())
print("before consume")
print(next(pipeline))
print("after first")
print(list(pipeline))

Nothing runs until the consumer pulls

close() runs finally

Calling close() raises GeneratorExit at the paused yield and runs finally blocks. That is how a generator releases a file or lock when a consumer stops early. A with block is usually clearer, but close() explains the cleanup contract.

def resource():
    try:
        yield "open"
        yield "still open"
    finally:
        print("cleanup")

gen = resource()
print(next(gen))
gen.close()
print("closed")

Early stop still reaches cleanup

Try it yourself

Change the values, predict the output, then run the code. The Python runtime is downloaded the first time you press Run, so the first run can take a few seconds.