Iterables, iterators, generators, oh my!

What is a Python iterable?

According to the Python official documentation:

An object capable of returning its members one at a time. Examples of iterables include all sequence types (such as list, str, and tuple) and some non-sequence types like dict, file objects, and objects of any classes you define with an __iter__() method or with a __getitem__()method that implements Sequence semantics.

What then is an iterator?

An object representing a stream of data. Repeated calls to the iterator’s __next__() method (or passing it to the built-in function next()) return successive items in the stream.

I am just shamelessly lifting up the definitions from the Python documentation. Don’t let that bother you 🙂

What is the difference/relationship between iterable and iterator?

Well, passing an iterable to the iter() function produces an iterator, a stream of data

that you can access  it’s items one by one( by calling the iterator’s __next__() method).

This is what happens when you pass an iterable through a for loop for example. Behind the scenes, the for statement calls iter() on the container object, which (as noted already) creates an iterator object that implements the __next__() method.

What is a generator?

It is a function that creates/returns an iterator.  Wait, I can explain. Normally, to create an iterator, you will need to implement the iterator protocol. That means you need a class that defines an __iter__() method which returns an object with a __next__() method.  Something like this(lifted again from the official doc):

class Reverse:
    """Iterator for looping over a sequence backwards."""
    def __init__(self, data):
        self.data = data
        self.index = len(data)

    def __iter__(self):
        return self

    def __next__(self):
        if self.index == 0:
            raise StopIteration
        self.index = self.index - 1
        return self.data[self.index]

In general, generators do the same thing. Except for two cases:

1. In a generator, the __iter__() and __next__() methods are created automatically.
2. In a generator,  local variables and execution state are automatically saved between calls. What this implies is that each time thenext() method is called on a generator iterator, it resumes from where it left off.

An example generator:

def reverse(data):
    for index in range(len(data)-1, -1, -1):
        yield data[index]

 

What is a generator expression.

An expression that returns an iterator. An example of a generator expression:

(i*i for i in range(10)