from Hacker News

A look at some of Python's useful itertools

by naiquevin on 5/25/13, 10:17 AM with 28 comments

• by wting on 5/25/13, 1:38 PM

  My intention is not to be snarky, but people post all the time about discovering the itertools or collections library. I notice it's a common gap in newer Python programmers.

  Save yourself time and effort down the road and read through both libraries' documentation, they're well worth the effort:

  http://docs.python.org/3.3/library/itertools.html

  http://docs.python.org/3.3/library/collections.html

  I tend to use defaultdict, deque (thread safe), namedtuple, imap, izip, drop/takewhile. In Python 3, map and zip have been replaced with their itertools equivalents.

  I blame Haskell for all the lazy evaluation influence. :P

• by masklinn on 5/25/13, 3:06 PM

      def flatmap(f, items):
          return itertools.chain(*map(f, items))
  

  1. in Python 2 `map` is eager which — as with the previous `even` filter — may lead to unnecessary work if you only need part of the list (or a dead process if the input is infinite...). itertools.imap (or a generator comprehension) would be better. This is "fixed" in Python 3 (where the `map` builtin has become lazy and `itertools.imap` has been removed) but

  2. it's being eagerly unpacked through *, itertools.chain also provides a from_iterable method which doesn't have that issue (and can be used to flatten infinite streams), introduced in 2.6

  So `flatmap` would probably be better as:

      def flatmap(f, items):
          return itertools.chain.from_iterable(
              itertools.imap(
                  f, items))

• by serjeem on 5/25/13, 4:28 PM

  I wrote my favorite function ever last semester with itertools! It (roughly) lazily generates a list of dictionaries that map players to their moves for all possible moves. It turns out you can do that with a chain of combinations, two cartesian products, and an imap: https://github.com/shargoj/acquire/blob/master/gametree.py#L...

• by davvolun on 6/3/13, 4:27 PM

  On the other hand, I suspect some early programmers might get ahold of this and perform a lot of premature optimizations. A piece of code that runs 20 loops instead of 8 once every couple of hours probably doesn't need to be optimized. A piece of code that does two checks when one would suffice that runs 1000 times every second might need optimization. Profile first, then optimize.