What are Python generators good for?

Image for post
Image for post
Flow of … data?

One of best features in Python (IMHO) is the “generator” thing.

Here’s a simple example of a generator in action:

The function produces values and handles it to a “for loop” as a collection. The obvious use is to avoid creating whole arrays and fill up memory.

I like them for another reason; they allow me to remove nesting. Recently I created a simple CSV from a query in Neo4J. The code goes like this:

Not pretty, the CSV (consumer) and Neo4J (producer) logics are interleaved. Generators to the rescue:

I find this better from a reuse perspective. The logic from producer and consumer are separated now. The producer function can generate records for any use. Also, I can use the write_to_csv function to write any collection of hash-like things to csv.

However, it’s a bit longer than the previous version and a little harder to understand, as the reader needs to know about generators. So, the second version is not good as throwaway code.

There is an important caveat to notice, also, if you don’t go over all records in the “cypher” generator, none of the Neo4J stuff will be closed, and you’ll get a connection leak for free. Be careful.

Written by

Just a common guy

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store