Type something to search...
functools in Python: partial, lru_cache, reduce, and wraps

functools in Python: partial, lru_cache, reduce, and wraps

Python treats functions as ordinary values. You can pass them around, store them in dicts, return them from other functions, and wrap them in new behavior. The functools module is the standard library's toolbox for doing exactly that: it holds "higher-order" helpers that take functions and give you back new, improved functions.

Four of them show up constantly in real code: partial for pre-filling arguments, lru_cache (and its sibling cache) for memoization, reduce for folding a sequence down to one value, and wraps for writing decorators that don't break introspection. This post covers each one in depth, including the edge cases that cause real bugs, and finishes with a quick look at cached_property, total_ordering, and singledispatch.

All examples run on Python 3.13.

partial: Pre-Filling Function Arguments

partial(func, *args, **kwargs) returns a new callable that behaves like func with some arguments already supplied. When you call it, the stored arguments are combined with whatever you pass in.

from functools import partial


def power(base: float, exponent: float) -> float:
    return base ** exponent


square = partial(power, exponent=2)
cube = partial(power, exponent=3)

print(square(5), cube(2))
25 8

The resulting object is a functools.partial instance. It keeps track of what it wraps, which makes it easy to debug:

print(square.func, square.args, square.keywords)
# <function power at 0x...> () {'exponent': 2}

Keyword arguments stored in a partial are defaults, so a caller can override them: square(5, exponent=4) returns 625.

Practical Uses

partial really shines when an API expects a callable with a specific signature and your function needs extra configuration.

Converting binary strings with int:

basetwo = partial(int, base=2)
print(basetwo("10010"))  # 18

Creating a preconfigured JSON serializer you can pass around:

import json
from functools import partial

dumps_pretty = partial(json.dumps, indent=2, sort_keys=True)
print(dumps_pretty({"b": 1, "a": 2}))
{
  "a": 2,
  "b": 1
}

Supplying a key or callback that needs an extra argument, for example with sorted, map, thread pools, or GUI and event-loop callbacks:

from concurrent.futures import ThreadPoolExecutor
from functools import partial

import urllib.request


def fetch(url: str, timeout: float) -> int:
    with urllib.request.urlopen(url, timeout=timeout) as resp:
        return resp.status


urls = ["https://www.python.org", "https://docs.python.org"]
with ThreadPoolExecutor() as pool:
    statuses = list(pool.map(partial(fetch, timeout=5), urls))

pool.map calls its function with one argument per item. partial(fetch, timeout=5) turns the two-argument fetch into the one-argument function it needs.

Positional Arguments Fill from the Left

Positional arguments given to partial are placed before the caller's arguments:

def greet(greeting: str, name: str) -> str:
    return f"{greeting}, {name}!"


hello = partial(greet, "Hello")
print(hello("Ada"))  # Hello, Ada!

That means you can only pre-fill leading positional parameters. To fix a later one, use a keyword. But watch out for mixing styles: if you freeze exponent by keyword and then call square(2, 3), the 3 also lands in exponent and Python raises TypeError: power() got multiple values for argument 'exponent'.

partial vs lambda

You could write square = lambda b: power(b, exponent=2) instead. Both work, but partial has two advantages:

  • It evaluates arguments immediately. A lambda looks up variables when it's called, which leads to the classic late-binding bug in loops:
from functools import partial

callbacks_l = [lambda: i for i in range(3)]
callbacks_p = [partial(print, i) for i in range(3)]

print([c() for c in callbacks_l])   # [2, 2, 2]
for c in callbacks_p:
    c()                             # prints 0, 1, 2
  • It's introspectable and picklable (as long as the wrapped function is), which matters when you send work to other processes with multiprocessing.

Lambdas are still fine when you need to rearrange arguments or add logic. There's more on that trade-off in the post on lambda functions.

There's also partialmethod, which does the same thing for methods defined in a class body, so self is still handled correctly.

lru_cache and cache: Memoization in One Line

Memoization means remembering a function's results so that repeated calls with the same arguments return instantly. lru_cache adds a cache to any function whose result depends only on its arguments:

from functools import lru_cache


@lru_cache(maxsize=None)
def fib(n: int) -> int:
    return n if n < 2 else fib(n - 1) + fib(n - 2)


print(fib(100))
print(fib.cache_info())
354224848179261915075
CacheInfo(hits=98, misses=101, maxsize=None, currsize=101)

Without the cache, fib(100) would make an astronomical number of calls and never finish. With it, each value from 0 to 100 is computed exactly once (the 101 misses), and every other lookup is a cache hit.

@cache (Python 3.9+) is shorthand for @lru_cache(maxsize=None): an unbounded cache with slightly less overhead.

maxsize and Eviction

LRU stands for "least recently used". With a bounded maxsize, once the cache is full, the entry that hasn't been used for the longest time is thrown out. The default is maxsize=128, and you can write the decorator either as @lru_cache or @lru_cache().

@lru_cache(maxsize=2)
def slow_square(n: int) -> int:
    print(f"computing {n}")
    return n * n


slow_square(2)   # computing 2
slow_square(3)   # computing 3
slow_square(2)   # cache hit, nothing printed
slow_square(4)   # computing 4  (evicts 3, the least recently used)
slow_square(3)   # computing 3  (it was evicted)
print(slow_square.cache_info())
CacheInfo(hits=1, misses=4, maxsize=2, currsize=2)

Pick a bound for anything that could see many distinct arguments over the life of a process. An unbounded @cache on a function called with user IDs is a slow memory leak.

Managing the Cache

Every cached function gets two extra methods:

  • cache_info() returns hits, misses, max size, and current size. Check it in a REPL or log it to see whether your cache is helping.
  • cache_clear() empties the cache. Call it when the underlying data changes, or between tests so results don't leak across them.

The original, uncached function is available as fib.__wrapped__.

Rules and Gotchas

Arguments must be hashable. The cache is a dict keyed on the arguments, so lists, dicts, and sets don't work:

from functools import cache


@cache
def total(items):
    return sum(items)


total([1, 2, 3])
# TypeError: unhashable type: 'list'

Pass a tuple or frozenset instead, or convert inside a thin wrapper.

Only cache pure functions. If the result depends on the current time, a database, a file, or a random number, the cache will happily return stale results. It also skips side effects on hits: a cached function that sends an email only sends it once per distinct argument.

Returned objects are shared. If a cached function returns a list and a caller mutates it, every future caller gets the mutated list. Return immutable values (tuples, frozen dataclasses) from cached functions, or copy them.

Keyword order matters. f(a=1, b=2) and f(b=2, a=1) may be cached as separate entries, and so may f(1) and f(a=1). It's still correct, just less efficient. Similarly, typed=True guarantees that f(1) and f(1.0) are cached separately; without it, the docs only promise they'll usually be treated as the same call.

Be careful on methods. Putting @lru_cache on an instance method includes self in the key, which means the cache holds a strong reference to every instance it's seen, keeping them alive as long as the cache does. For per-instance values, prefer cached_property (below). For methods that don't use self, make them a module-level function or a staticmethod and cache that.

Thread safety. The cache's internal data structure stays consistent across threads, but two threads can call the function with the same new argument at the same time and both compute it. That's usually harmless; just don't rely on "computed exactly once".

reduce: Folding a Sequence into One Value

reduce(function, iterable[, initial]) applies a two-argument function cumulatively: it combines the first two items, then combines that result with the third, and so on.

import operator
from functools import reduce

print(reduce(operator.add, [1, 2, 3, 4]))                    # ((1+2)+3)+4 = 10
print(reduce(lambda acc, x: acc * x, [1, 2, 3, 4], 1))       # 24

The optional third argument is the starting value. Always pass it when the input could be empty, because without it reduce has nothing to return:

reduce(operator.add, [])
# TypeError: reduce() of empty iterable with no initial value

reduce(operator.add, [], 0)   # 0

When reduce Is the Right Tool

Honestly, not often. Guido van Rossum moved reduce out of the builtins in Python 3 precisely because most uses are clearer as a loop or a dedicated builtin:

Instead ofUse
reduce(operator.add, nums)sum(nums)
reduce(operator.mul, nums)math.prod(nums)
reduce(max, nums)max(nums)
reduce(lambda a, b: a and b, flags)all(flags)
reduce(math.gcd, nums)math.gcd(*nums) (3.9+)

Where reduce does read well is when the combining step is a real operation with a name and no builtin covers it. Walking a nested dict by a list of keys is a good example:

import operator
from functools import reduce

config = {"db": {"primary": {"host": "10.0.0.5"}}}
print(reduce(operator.getitem, ["db", "primary", "host"], config))  # 10.0.0.5

Composing a list of functions into one pipeline is another:

from functools import reduce


def compose(*funcs):
    return reduce(lambda f, g: lambda x: g(f(x)), funcs)


slugify = compose(str.strip, str.lower, lambda s: s.replace(" ", "-"))
print(slugify("  Hello World  "))  # hello-world

If you have to stare at a reduce call to understand it, rewrite it as a for loop with an accumulator. It'll be the same speed and far easier to read.

wraps: Writing Decorators That Behave

A decorator replaces a function with a wrapper. The problem is that the wrapper is a different function, with its own name and docstring:

def timed(func):
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper


@timed
def add(a: int, b: int) -> int:
    """Add two numbers."""
    return a + b


print(add.__name__, add.__doc__)  # wrapper None

Now every decorated function in your logs, stack traces, help() output, and test reports shows up as wrapper. Tools that read signatures or annotations (FastAPI, Typer, pytest fixtures, documentation generators) see the wrong thing too.

@wraps(func) fixes this by copying the original function's metadata onto the wrapper:

import time
from functools import wraps


def timed(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        try:
            return func(*args, **kwargs)
        finally:
            elapsed = time.perf_counter() - start
            print(f"{func.__name__} took {elapsed:.4f}s")
    return wrapper


@timed
def add(a: int, b: int) -> int:
    """Add two numbers."""
    return a + b


print(add(1, 2))
print(add.__name__, add.__doc__)
print(add.__annotations__)
add took 0.0000s
3
add Add two numbers.
{'a': <class 'int'>, 'b': <class 'int'>, 'return': <class 'int'>}

Specifically, wraps copies __module__, __name__, __qualname__, __doc__, __annotations__, and (since 3.12) __type_params__, merges the original's __dict__ into the wrapper's, and sets __wrapped__ to point at the original function. inspect.signature() follows __wrapped__, so the decorated function reports the original signature (a: int, b: int) -> int instead of (*args, **kwargs).

__wrapped__ is also an escape hatch in tests: add.__wrapped__(3, 4) calls the undecorated function directly.

The rule is simple: every decorator you write should use @wraps on its inner function. It costs one line. The full story of how decorators work, including decorators with arguments, is in Python decorators explained.

Other functools Helpers Worth Knowing

cached_property

Computes a value the first time it's accessed, then stores it on the instance as a normal attribute:

from functools import cached_property


class Report:
    def __init__(self, rows: list[int]) -> None:
        self.rows = rows

    @cached_property
    def total(self) -> int:
        print("summing...")
        return sum(self.rows)


r = Report([1, 2, 3])
print(r.total)   # summing... then 6
print(r.total)   # 6, no recomputation
del r.total      # clears the cached value
print(r.total)   # summing... then 6

Because the value lives in the instance's __dict__, it's freed when the instance is, which is why it's the better choice over lru_cache for per-object computations. It requires a __dict__, so it doesn't work on classes that use __slots__ without one.

total_ordering

Define __eq__ and one of __lt__, __le__, __gt__, or __ge__, and @total_ordering fills in the rest:

from functools import total_ordering


@total_ordering
class Version:
    def __init__(self, major: int, minor: int) -> None:
        self.major, self.minor = major, minor

    def __eq__(self, other):
        return (self.major, self.minor) == (other.major, other.minor)

    def __lt__(self, other):
        return (self.major, self.minor) < (other.major, other.minor)


print(Version(1, 2) >= Version(1, 1), Version(2, 0) <= Version(1, 9))  # True False

For plain data classes, @dataclass(order=True) does the same job with less code.

singledispatch

Turns a function into a generic function that picks an implementation based on the type of its first argument:

from functools import singledispatch


@singledispatch
def describe(obj) -> str:
    return f"object: {obj!r}"


@describe.register
def _(obj: int) -> str:
    return f"int: {obj}"


@describe.register
def _(obj: list) -> str:
    return f"list of {len(obj)}"


print(describe(3), describe([1, 2]), describe("x"), describe(True))
int: 3 list of 2 object: 'x' int: True

Note the last one: bool is a subclass of int, so True uses the int implementation unless you register bool separately. For methods, use singledispatchmethod.

Quick Reference

ToolWhat it doesReach for it when
partialPre-fills argumentsAn API wants a callable with fewer parameters
lru_cache / cacheMemoizes resultsA pure function is called repeatedly with the same arguments
reduceFolds an iterable into one valueNo builtin like sum/max/all fits
wrapsCopies metadata onto a wrapperEvery time you write a decorator
cached_propertyComputes an attribute once per instanceExpensive derived attributes
total_orderingFills in comparison methodsCustom classes that need sorting
singledispatchDispatches on argument typeOne operation, several input types

Conclusion

functools is small, but nearly every tool in it removes a whole category of boilerplate. Use partial when you need a function with some arguments locked in, and prefer it over a lambda inside loops. Use lru_cache or cache on pure functions with hashable arguments, and give the cache a bound when inputs are unbounded. Use reduce sparingly, only when no builtin expresses the fold more clearly. And put @wraps in every decorator you write, so your functions keep their names, docs, and signatures.

Tags :
Share :

Related Posts

Abstract Base Classes in Python with the abc Module

Abstract Base Classes in Python with the abc Module

Python leans on duck typing: if an object has the method you need, you call it and move on. That works well until you have a family of classes that a

Continue Reading
*args and **kwargs in Python: Flexible Function Signatures

*args and **kwargs in Python: Flexible Function Signatures

You've seen def wrapper(*args, **kwargs): in decorators, and probably super().__init__(**kwargs) in class hierarchies. These two parameters let a

Continue Reading
Asyncio in Python: A Beginner's Guide to Asynchronous Programming

Asyncio in Python: A Beginner's Guide to Asynchronous Programming

A lot of programs spend most of their time waiting. A web scraper waits for pages to download, an API server waits for the database, a chat bot waits

Continue Reading