How do I use itertools to work with iterables efficiently in Python?

The short version

The itertools module provides lazy tools for combining and slicing iterables: chain joins them, islice takes a slice, combinations and permutations generate groupings, and groupby clusters consecutive equal items. Because they yield items on demand, they use little memory and compose cleanly, which makes them ideal for large or streaming data.

Joining iterables with chain

from itertools import chain

print(list(chain([1, 2], [3, 4], [5])))   # [1, 2, 3, 4, 5]

chain treats several iterables as one continuous sequence without building a combined list, so it is memory-cheap even across many large inputs.

Slicing with islice

islice takes a slice of any iterable, including generators and files that you cannot index.

from itertools import islice

first_three = list(islice(range(1000000), 3))   # [0, 1, 2], no big list built

This is how you take "the first N" from something you cannot slice with [:3], such as a generator or an open file.

Combinations and permutations

For generating groupings, these produce each result lazily rather than all at once.

from itertools import combinations

list(combinations([1, 2, 3], 2))   # [(1, 2), (1, 3), (2, 3)]

combinations ignores order (so (1, 2) but not (2, 1)), while permutations includes both orderings. Use these instead of nested loops when you need every pairing or grouping.

The groupby catch

itertools.groupby clusters consecutive equal items, which is different from a pandas or SQL group by. It only groups runs that are already adjacent, so you must sort first if you want all matching items together.

from itertools import groupby

data = [("a", 1), ("a", 2), ("b", 3), ("a", 4)]
for key, group in groupby(data, key=lambda x: x[0]):
    print(key, [v for _, v in group])

Because the second a is not adjacent to the first, it forms its own group. This surprises people expecting all a items together, so sort by the key first if that is what you want.

Why laziness matters

Each of these returns an iterator that produces values one at a time, so a chain of them never materialises the full intermediate results. On a large input this is the difference between a small, constant memory footprint and building several big lists. When you find yourself writing nested loops or building throwaway lists to combine sequences, an itertools function is often shorter and lighter.

A few more worth knowing

count, cycle and repeat create infinite iterators (use with islice to bound them), accumulate gives running totals, and product gives the Cartesian product of several iterables. Reaching for these named tools makes intent clear and avoids hand-rolling the same logic with loops.