Object decorators are objects that extend and customize functions in Python, and since they are objects themselves, they can store information that can be modified independently of the decorated function. This article explains the basics of Python decorators and how custom objects can be used as decorators.

Decorators are functions (or objects) that wrap other functions, which simply means that they execute commands before and after the function that is wrapped. Decorators are useful because they extend the functionality of existing functions by factoring out pieces of code that do not pertain to the purpose of the function.

In Python, decorator functions have the following syntax:

def my_decorator(function):
    """A instructive decorator."""

    def wrapper(*args, **kwargs):
        print "This is executed before {}".format(function.__name__)
        return_value = function(*args, **kwargs)
        print "This is executed after {}".format(function.__name__)
        return return_value

    return wrapper

@my_decorator
def sum(a,b):
    return a+b

result = sum(2, b=3)
print result

The decorator (or wrapper) function, in this case, is named my_decorator, and the decorated (or wrapped) function is named sum. The sum function’s purpose is simply to add two numbers, and the my_decorator simply prints a statement before and after execution, presumably for debugging purposes. Keeping the decorator function separate removes the clutter of print statements in the sum function, and it allows my_decorator to be used for other functions.

Executing the code above gives the following:

This is executed before sum
This is executed after sum
5

As you might have noticed, the behavior of the above decorator cannot be directly customized–nor would it need to be for this simple decorator. In the following sections, I will demonstrate two different methods of customizing decorator behavior using a memoizer as an example.

Object Decorators: The Memoizer

A memoizer is a function (decorator) that stores the results of the wrapped function. It is a form of function optimization that caches computationally-expensive results into memory, returning these on future invocations rather than re-evaluating the answer every time. Non-trivial uses might include database lookup functions or expensive mathematical calculations with values that depend only on the specified arguments (holonomic, state functions).

Object decorators are particularly useful as memoizers because they can manage their own resources, and they can incorporate special methods for cleaning or deleting the cached data. A maintenance function could be used, for example, to remove values stored by the memoizer for which the result is known to have changed, as would be the case for a function that returns the number of records in a ever-changing database.

Let me illustrate with an example.

class Memoizer(object):
    """Caches the result of a function.
    """

    def __init__(self, function):
        self.function = function
        self.__name__ = function.__name__
        self.__doc__ = function.__doc__

        self.cached_values = {}

    def __call__(self, *fn_args,**fn_kwargs):
        """Wrapper for calling the Cached Function.
        """
        assert not fn_args, ("CachedMethod cannot be used with positional"
                             "arguments.")

        kwargs_sig = self.kwargs_signature(**fn_kwargs)

        if kwargs_sig not in self.cached_values:
            print "Calculating new value."
            self.cached_values[kwargs_sig] = self.function(**fn_kwargs )
        else:
            print "Returning cached value."

        return self.cached_values[kwargs_sig]

    def kwargs_signature(self, **fn_kwargs):
        """Returns kwargs signature--an immutable, given a set of kwargs.
        """
        tp = tuple([v for k,v in sorted(fn_kwargs.items())])
        return str(tp.__hash__())

    def clear(self, **fn_kwargs):
        """Clears the cached data."""

        if not fn_kwargs:
            # Clear all if a set of kwargs was not specified.
            self.cached_values = {}
        else:
            # Otherwise just clear the value for the specified kwargs
            # combination.
            kwargs_sig = self.kwargs_signature(**fn_kwargs)
            self.cached_values.pop(kwargs_sig, None)

The above Memoizer stores the memoized function (self.function) and a dict containing the cached results (self.cached_values). The __call__ method implements the behavior of this object when it is called like a function, including arguments and keyword arguments. When a Memoizer object wraps a function, the __call__ method is executed, and it is tasked with executing the wrapped function then storing the result. A clear method has been included which can either be used to remove all cached values, when invoked without parameters, or to remove specific cached values, when invoked with parameters.

A technical note on arguments: efficient storage of memoized results requires that a given set of arguments corresponds to only one result. The combination of parameters used in invoking a function is known as the signature of a function. Due to the Python’s flexibility with arguments, a function can be executed with the same parameters but with different signatures.

sum(3, b=1)
sum(a=3, b=1)
sum(3,1)

These three statements give the same arguments to the sum function and return the same result, but they all have different (but valid!) signatures. This is a problem when using *args and **kwargs to abstract the function arguments: given the flexibility of writing arguments with keywords (sum(a=3, b=1)), without keywords (sum(3,1)), or a mixture of the two cases (sum(3, b=1)), the given parameters will jump between the args tuple (when invoked without a keyword) and the kwargs dict (when invoked with a keyword). This makes it difficult to store one result for all three of these cases. The trade-off I’ve chosen is to assert that arguments without keywords are not used, and to transform the keyword arguments into a unique identifier (kwargs_signature) that can be used as a key to the self.cached_values dict. This approach arguably makes code clearer by requiring keywords, and it ensures that only one result is stored for a given set of parameters.

The Memoizer decorates functions like so:

@Memoizer
def sum(a, b):
    return a+b

print sum(a=3,b=2)    # Function is evaluated, return value
print sum(a=3,b=2)    # Function returns cached value
sum.clear()           # The cached value is cleared
print sum(a=3,b=2)    # Function is evaluated, return value

The result of the of sum function is stored in the Memoizer’s self.cached_values, and the stored values are deleted using the clear() method.

Decorator Arguments

Decorator arguments customize the behavior of decorators by allowing initialization parameters to change the behavior of the decorator when the decorator is created. In this example, the Memoizer is extended to measure the execution time of the wrapped function.

def Memoizer(*args, **kwargs):
    """The outer wrapper for creating a object decorated."""
    import time

    class MemoizerObject(object):
        """Caches the result of a function.
        """

        def __init__(self, function, verbose=True):
            self.function = function
            self.__name__ = function.__name__
            self.__doc__ = function.__doc__

            self.verbose = verbose

            self.cached_values = {}
            self.statistics = {}

        def __call__(self, *fn_args, **fn_kwargs):
            """Wrapper for calling the Cached Function.
            """
            assert not fn_args, ("CachedMethod cannot be used with positional "
                              "arguments.")

            kwargs_sig = self.kwargs_signature(**fn_kwargs)

            # Calculate the new value, if needed and update the functional call
            # statistics
            if kwargs_sig not in self.cached_values:
                t1 = time.time()
                self.cached_values[kwargs_sig] = self.function(**fn_kwargs )
                t2 = time.time()

                # Update the statistics
                self.statistics['time_last_call'] = (t2-t1)*1000. # milliseconds
                self.statistics['number_of_calls'] = \
                    self.statistics.get('number_of_calls', 0) + 1

                if self.verbose:
                    print ("'{name}' calculated a new value which took "
                           " {time:.3f} ms.".format(name=self.__name__,
                           time=self.statistics['time_last_call']))
            else:
                if self.verbose:
                    print ("'{name}' returned a cached "
                           "value.".format(name=self.__name__))

            return self.cached_values[kwargs_sig]

        def kwargs_signature(self, **fn_kwargs):
            """Returns kwargs signature--an immutable, given a set of kwargs.
            """
            tp = tuple([v for k,v in sorted(fn_kwargs.items())])
            return str(tp.__hash__())

        def clear(self, **fn_kwargs):
            """Clears the cached data and statistics."""

            if not fn_kwargs:
                # Clear all if a set of kwargs was not specified.
                self.cached_values = {}
            else:
                # Otherwise just clear the value for the specified kwargs
                # combination.
                kwargs_sig = self.kwargs_signature(**fn_kwargs)
                self.cached_values.pop(kwargs_sig, None)

            # Clear the statistics
            self.statistics.clear()

    def inner_decorator(function):
        return MemoizerObject(function, *args, **kwargs)
    return inner_decorator

A key difference in this implementation from the former is the use of an outer decorator function. The Memoizer is now a function that creates a MemoizerObject instance to be used in wrapping the target function. The outer Memoizer function customizes the creation of the ```MemoizerObject`` with the decorator arguments, and it returns a decorator that wraps the function with this object.

The extra level of abstraction with the outer decorator is needed because the wrapped function is not given as a parameter to the MemoizerObject constructor function (__init__), and therefore it cannot be used directly to wrap the function.

This new decorator object collects statistics on the execution time and the number of calls to the wrapped function, and these are reported when the verbose option is set to True.

@Memoizer(verbose=True)
def sum(a, b):
    return a+b

print sum(a=3,b=2)    # Function is evaluated, return value
print sum(a=3,b=2)    # Function returns cached value
sum.clear()           # The cached value is cleared
print sum(a=3,b=2)    # Function is evaluated, return value
print "{name} has been called {number} times.".format(name=sum.__name__,
    number=sum.statistics['number_of_calls'])

The generated output gives statistics when running the sum function.

'sum' calculated a new value which took  0.007 ms.
5
'sum' returned a cached value.
5
'sum' calculated a new value which took  0.001 ms.
5
sum has been called 1 times.

In the next article on Object Decorators, descriptors are used as Method Object Decorators, opening new possibilities for membership testing and class-level customization of object decorators.