Python functools module
The functools module in the standard library provides functions and classes for manipulating functions. The primary tool defined in the module is the partial class, which as we will see in a while, allows the partial application of a function. This means that a callable object is created with some of the arguments already filled in, reducing the number of arguments that must be supplied to subsequent calls.
The module also provides a number of decorators which can be used to wrap functions and classes in order to extend them with additional capabilities.
Partial Objects
The partial class makes it possible to create a version of a callable object( e.g functions) with some of the arguments pre-filled.
Syntax:
partial(callable, *args, **kwargs)
Example
#import the functools module
import functools
myprint = functools.partial(print, sep = ', ')
#use the partial function
myprint(*range(10))
myprint('Python', 'C++', 'Java', 'Ruby', 'Javascript')
Output:
0, 1, 2, 3, 4, 5, 6, 7, 8, 9 Python, C++, Java, Ruby, Javascript
In the above example, we created a new partial function, myprint(), from the builtin print() function. The original print() function would have separated each item with a space, but by using functools.partial function, we were able to set the sep keyword argument to a comma and a space( ', ').
In the following example we use a user-defined function to create a partial function.
Example
import functools
def evaluate(num1, num2, oper):
"""Evaluates the arithmetic result of applying operator, 'oper' on 'num1' and 'num2'"""
num1, num2 = int(num1), int(num2)
result = None
match oper:
case "+":
result = num1 + num2
case "-":
result = num1 - num2
case "*":
result = num1 * num2
case "/":
result = num1 / num2
case "%":
result = num1 % num2
case "**":
result = num1 ** num2
case _:
return ("Invalid operator '%s'"%oper)
return f"{num1} {oper} {num2} = {result}"
add = functools.partial(evaluate, oper = '+')
print(add(3, 5))
print(add(10, 20))
print(add(50, 60))
Output:
3 + 5 = 8 10 + 20 = 30 50 + 60 = 110
The following example uses a class rather than a function as the callable
Example
import functools
class Person:
def __init__(self, name, country, nationality):
self.name = name
self.country = country
self.nationality = nationality
def info(self):
return f"""name: {self.name}
country: {self.country}
nationality: {self.nationality}"""
#create a partial constructor for people of a specific country
indian = functools.partial(Person, country = 'India', nationality = "Indian")
p1 = indian('Rahul')
print(p1.info())
Output:
name: Rahul country: India nationality: Indian
Acquire properties of the original callable
By default, the returned partial object, does not inherit the __name__ and the __doc__ attributes from the original callable object.This attributes are very essential especially for debugging purposes.
Example
import functools
myprint = functools.partial(print, sep = ', ')
#get the __doc__ attribute
print(myprint.__doc__)
Output:
partial(func, args, *keywords) - new function with partial application of the given arguments and keywords.
As you can see above, myprint() function does not inherit the docstring of the print() function, from which it is a partial object.
The update_wrapper() function attaches the relevant information to the partial object from another object.
Example
import functools
myprint = functools.partial(print, sep = ', ')
functools.update_wrapper(myprint, print)
#get the __doc__ attribute
print(myprint.__doc__)
Output:
Prints the values to a stream, or to sys.stdout by default. sep string inserted between values, default a space. end string appended after the last value, default a newline. file a file-like object (stream); defaults to the current sys.stdout. flush whether to forcibly flush the stream.
partialmethod class
The partial class, as we have seen, works with bare functions. The partialmethod class similarly creates partial objects but from methods. The partial method should be defined inside the class as shown below.
Example
import functools
class Person:
def __init__(self, name, country, nationality):
self.name = name
self.country = country
self.nationality = nationality
def change_nationality(self, new_country, new_nationality):
self.country = new_country
self.nationality = new_nationality
#create a partial metthod for 'change_nationality'
to_japanese = functools.partialmethod(change_nationality, new_country = 'Japan', new_nationality = 'Japanese')
p1 = Person('Rahul', 'India', 'Indian')
print(p1.name, p1.country, p1.nationality)
#call the partialmethod 'to_japanese' on object
p1.to_japanese()
print(p1.name, p1.country, p1.nationality)
Output:
Rahul India Indian Rahul Japan Japanese
Functions defined in the functools module
Apart from the two classes that we have looked at i.e partial and partialmethod, the module also defines several utility functions that can be used to further manipulate functions through decoration.
cmp_to_key()
Some high order functions such builtin sorted(), filter(), max(), and min(), takes an optional parameter called key which is used to specify a particular function to be applied to each element of the iterable prior to making comparisons.
Example
L = [-10, -2, 5, 0, -1, 4]
#the key function
def absolute(x):
if x < 0:
return -x
return x
#sort the elements by their absolute value
print(sorted(L, key = absolute))
Output:
[0, -1, -2, 4, 5, -10]
The cmp_to_key() function is used to create a key function from a traditional comparison function. The function given as an argument must return either 1, 0, or -1 depending on the arguments it receives.
Example
import functools
L = [('Python', 3), ('Java', 4), ('C++', 2), ('Javascript', 1), ('PHP', 5)]
# function to Sort the tuples by their second items
@functools.cmp_to_key
def func(a, b):
if a[-1] > b[-1]:
return 1
elif a[-1] < b[-1]:
return -1
else:
return 0
#sort the elements by their last element
print(sorted(L, key = func))
Output:
[('Javascript', 1), ('C++', 2), ('Python', 3), ('Java', 4), ('PHP', 5)]
reduce()
The reduce() function applies a given function cumulatively to an iterable. It typically takes two arguments: a function and an iterable, it then applies the function cumulatively on the elements of the given iterable.
Syntax:
reduce(func, iterable)
The function given as func must accept two arguments.
Example
import functools
L = [1, 2, 3, 4, 5, 6, 7, 8, 9]
def add(a, b):
return a + b
cumulative_sum = functools.reduce(add, L)
print(cumulative_sum)
Output:
45
Example
import functools
L = [1, 2, 3, 4, 5, 6, 7, 8, 9]
def prod(a, b):
return a * b
cumulative_prod = functools.reduce(prod, L)
print(cumulative_prod)
Output:
362880
cache()
The cache() function is used to cache the result of an expensive computation for future use. This means that if the same function is called with the same parameters, the results are cached and the computation does not need to be redone.
Example
import functools
@functools.cache
def fibonacci(num):
if num in [0, 1]:
return num
return fibonacci(num - 1) + fibonacci(num - 2)
# Let's run it and check the cache
print(fibonacci(10))
print(fibonacci(10))
Output:
55 55
lru_cache()
The lru_cache() function implements a least recently used (LRU) cache for efficient memoization of a function. When the function is called, the lru_cache() will store any inputs and outputs of the function in an order of least recently used. When the cache is full, the least recently used items are discarded to make space for new data. This is beneficial as it allows the cache to store more relevant data rather than having to store all data from the function call.
Example
import functools
@functools.lru_cache(maxsize=4)
def fibonacci(num):
if num in [0, 1]:
return num
return fibonacci(num - 1) + fibonacci(num - 2)
# Let's run it and check the cache
print(fibonacci(10))
print(fibonacci(10))
Output:
55 55
singledispatch()
The singledispatch() decorator function is used to create functions that can dispatch on the type of a single argument such that certain behaviors are dependent on the type. When decorated with singledispatch(), a function becomes a "generic function", meaning that it can have multiple different implementations, depending on the type of the argument passed to it.
The implementation for a particular type is registered using the register() method of the decorated function.
Example
import functools
@functools.singledispatch
def add(a, b):
raise NotImplementedError
@add.register(str)
def _(a, b):
return f"{a} {b}"
@add.register(int)
def _(a, b):
return a + b
#with ints
print(add(1, 2))
#with strings
print(add("Hello", "World"))
#an error is raised for a non-implemented types
print(add([1, 2], [3, 4]))
Output:
3 Hello World NotImplementedError:
@total_ordering
The @functools.total_ordering() is used to automatically fill in comparison methods for a class by defining only two of the six rich comparison methods (__lt__, __le__, __eq__, __ne__, __gt__ and __ge__) . This is typically achieved by decorating the class with the @total_ordering decorator and defining the __eq__ method alongside any other of (__lt__, __le__,__gt__, __ge__).
Example
import functools
@functools.total_ordering
class Student:
def __init__(self, name, grade):
self.name = name
self.grade = grade
def __repr__(self):
return self.name
def __eq__(self, other):
return self.grade == other.grade
def __lt__(self, other):
return self.grade < other.grade
john = Student("John", 83)
jane = Student("Jane", 87)
print(john == jane)
print(john != jane)
print(john < jane)
print(john <= jane)
print(john > jane)
print(john >= jane)
Output:
False True True True False False