Iterables versus iterators: the protocol behind for loops

What it is

The library reference documents two methods. An iterable provides __iter__(), returning an iterator; an iterator provides __next__(), returning the next item or raising StopIteration, and its own __iter__() returning itself. A for loop calls iter() once and next() repeatedly. The consequence: a container yields a fresh iterator on every iter() call, while an iterator handed to a second loop returns itself and appears empty because it is already exhausted. The reference adds that once __next__() has raised StopIteration it must continue to do so, and calls implementations that do not "broken". iter(callable, sentinel) builds an iterator that calls a function until it returns the sentinel, and next(it, default) returns a default instead of raising.

Why it matters

Functions that ask for "a sequence" often accept any iterable and then break when they iterate twice (len(), a validation pass followed by a processing pass, membership tests). Knowing which of the two is in hand, and which a signature promises, removes a whole class of empty-result bugs.

How to apply

• Annotate parameters as Iterable[T] for a single pass and Sequence[T] or Collection[T] when len, indexing or several passes are needed; when unsure, materialise once with list(xs) at the boundary.
• Implement __iter__ as a generator function on custom containers; each call yields a new independent iterator without a hand-written iterator class.
• Write an iterator class only when it needs state a generator cannot express (peek, rewind, external control), and let __iter__ return self.
• Read fixed-size chunks with iter(partial(f.read, 4096), b""); the built-in documentation shows this block-reader idiom.
• Fetch a first element safely with next(it, None); skip with itertools.islice.
• Materialise lazy inputs before retries or before validation that precedes use; both would otherwise consume the data.

Pitfalls

isinstance(x, Iterable) says nothing about re-iterability; a generator passes and still exhausts. Open files are iterators over lines: a second for line in f yields nothing without f.seek(0). map, filter, zip, enumerate and reversed return single-pass iterators, whereas range, dict views and other containers stay re-iterable. Since PEP 479, a StopIteration escaping a generator body becomes RuntimeError, so next() calls inside generators need a default or a try. A class with __getitem__ accepting integers from 0 is iterable through the older sequence protocol even without __iter__.