Iterators & Generators
Iterators
The iterator protocol — understanding __iter__ and __next__ methods, how for loops work internally, and building custom iterable sequences.
Interview: Common interview topic — understanding iterators demonstrates deep knowledge of Python internals and lazy evaluation.
An iterator is any object that implements the iterator protocol: it must have __iter__() (returns itself) and __next__() (returns the next value or raises StopIteration). Iterators are the engine behind Python's for loops, comprehensions, and many built-in functions.
The Iterator Protocol
__iter__(self)— must return the iterator object itself (usuallyreturn self)__next__(self)— returns the next value; raisesStopIterationwhen exhausted- Once an iterator is exhausted, it cannot be restarted — create a new one
iter(obj)calls__iter__(),next(obj)calls__next__()
How for Loops Work
- Step 1: Call
iter()on the iterable to get an iterator - Step 2: Repeatedly call
next()on the iterator - Step 3: Catch
StopIterationto end the loop - This is why any iterable can be used in a for loop
Custom Iterators
Build your own iterators by implementing both __iter__ and __next__ in a class:
- Track state in instance variables (current position, limits, etc.)
- Use
itertools.islice()to take a finite portion of infinite iterators - Iterators are memory-efficient — they produce one value at a time
Interview Insight
Be able to explain the difference between iterables and iterators, and implement a custom iterator from scratch. Know that iterators are single-pass (exhausted after one use) while iterables can produce fresh iterators.
Use Cases
Streaming large datasets — processing one record at a time without loading all into memory
Custom sequences — generating mathematical sequences, file lines, API pages
Lazy evaluation — computing values on demand rather than upfront
Infinite sequences — generating IDs, timestamps, or test data
Pipeline processing — chaining iterators for data transformation
Common Mistakes
Forgetting to raise StopIteration — causes infinite loops
Not returning self from __iter__ — breaks the iterator protocol
Trying to reuse an exhausted iterator — create a new one instead
Confusing iterators with iterables — iterators have __next__, iterables have __iter__
Storing all values in memory — defeats the purpose of lazy evaluation