Welcome, guest | Sign In | My Account | Store | Cart

This cookbook contains many recipes to memoize functions, however a recipe to memoize classes was missing. Using this recipe you can cache object creation, i.e. __new__ and __init__ methods are called only when needed. For a good use case, see the discussion around http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/413609

Python, 22 lines
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
# use the whatever memoize implementation you like
 
class Memoize(type):
    @memoize
    def __call__(cls, *args):
        return super(Memoize, cls).__call__(*args)

if __name__ == "__main__": # test
    
    class Object:
        __metaclass__ = Memoize
        def __init__(self, *args):
            print "object created with parameters %s" % str(args)


    o1 = Object(1) # create the first object
    o2 = Object(1) # return the already created object
    assert o1 is o2

    o3 = Object(1, 2) # create another object
    o4 = Object(1, 2) # return the already created object
    assert o3 is o4

The test case in the code should be clear: if you pass to class the same parameters, you will get the same object. __new__ and __init__ methods will NOT be called the second time you pass the same parameters. Everything works since __new__ and __init__ are called by the metaclass __call__ method which has been memoized.

You may use whatever implementation you like for the memoize decorator, there are many availables in this cookbook and in the PythonDecoratorLibrary in the Python Wiki. I also have written a pretty cool decorator module that allows you to define signature-preserving decorators:

http://www.phyast.pitt.edu/~micheles/python/decorator.zip

4 comments

Zoran Isailovski 18 years, 10 months ago  # | flag

Why use a metaclass? What's wrong with

class Object...
...
Object = memoize(Object)
...
o1 = Object(...)

It seems to me the metaclass is only there to attach a decorator to its __call__ method.

Michele Simionato (author) 18 years, 10 months ago  # | flag

depends on your memoize implementation. I am using a very picky "memoize" implementation that takes functions and returns functions with the same signature, so it does not work on classes. This is on purpose since I do not want to break introspection tools. You could define a "memoize" that takes a class and returns a subclass of the original one with memoization properties, or modifies both the __init__ and __new__ methods, or just returns a factory object which is not a class. There is balance between not breaking introspection tools and easy of implementation. I am trying very hard not to break introspection features and the metaclass is the most convenient way of reusing the "memoize" implementation in my decorator module.

Michele Simionato (author) 18 years, 10 months ago  # | flag

Does not play well with keyword arguments. I forgot to mention that the implementation based on overriding __call__ does not work when calling the constructor with keyword arguments, unless __call__ is overridden with the right signature, but then one should define a different metaclass for any different signature.

A workaround is to memoize __new__ and __init__ separately, making sure they have a consistent signature, but then the implementation becomes too complex for a recipe, so let me skip it ;) Email me if you interested.

Jerry W Barrington 10 years, 5 months ago  # | flag

With this recipe (and appropriate memoize function) is there any straightforward way to access the cache of created objects? And does each class memoized this way get a separate cache?

Created by Michele Simionato on Mon, 9 May 2005 (PSF)
Python recipes (4591)
Michele Simionato's recipes (12)

Required Modules

  • (none specified)

Other Information and Tasks