Inserting Code to Run on Attribute Access

A better solution would allow you to run code automatically on attribute access if needed. That’s one of the main roles of managed attributes—they provide ways to add attribute accessor logic after the fact. More generally, they support arbitrary attribute usage modes that go beyond simple data storage.

At various points in this book, we’ve met Python tools that allow our scripts to dynamically compute attribute values when fetching them and validate or change attribute values when storing them. In this chapter, we’re going to focus more deeply on the tools already introduced, explore other tools in this category, and study some larger use-case examples in this domain. Specifically, this chapter presents four accessor techniques:

1. The __getattr__ and __setattr__ methods, for handling undefined attribute fetches and all attribute assignments

2. The __getattribute__ method, for handling all attribute fetches

3. The property built-in, for specifying methods to handle access to a specific attribute

4. The __get__ and __set__ descriptor methods, for handling access to a specific method and the basis for other tools such as properties and slots

We met these tools in Chapters 30 and 32 briefly, and in some cases, hardly at all. As you’ll see here, all four techniques share goals to some degree, and it’s usually possible to code a given problem using any one of them.

That said, they also differ in some important ways. For example, the last two techniques listed here apply to specific attributes, whereas the first two are generic enough to be used by delegation-based proxy classes that must route arbitrary attributes to wrapped objects. As you’ll find, all four schemes also differ in both complexity and aesthetics in ways you must see in action to judge for yourself.

Besides studying the specifics behind these four attribute interception techniques, this chapter also presents an opportunity to explore programs larger than most we’ve seen elsewhere in this book. The CardHolder case study at the end, for example, should serve as a self-study example of larger classes in action. We’ll also be using some of the techniques outlined here in the next chapter to code decorators, so be sure you have at least a general understanding of these topics before you move on.

The Basics

A property is created by assigning the result of a built-in function to a class attribute:

attribute = property(fget, fset, fdel, doc)

None of this built-in’s arguments are required, and all default to None if not passed. For the first three, this None means that the corresponding operation is not supported, and attempting it will raise an AttributeError exception automatically.

When these arguments are used, we pass fget a function for intercepting attribute fetches, fset a function for assignments, and fdel a function for attribute deletions. Technically, all three of these arguments accept any callable, including a class’s method, having a first argument to receive the instance being qualified. When later invoked, the fget function returns the computed attribute value, fset and fdel return nothing (really, None), and all three may raise exceptions to reject access requests.

The doc argument receives a documentation string for the attribute if desired. If omitted, the property copies the docstring of the fget function, which, as usual, defaults to None.

This built-in property call returns a property object, which we assign to the name of the attribute to be managed in the class scope, where it will be inherited by every instance. As you’ll learn ahead, this assignment can be automated by @ decorator syntax, though its distributed usage may seem awkward for set and delete methods. However assigned, later accesses to the attribute automatically invoke the property’s handlers.

A First Example

To demonstrate how this translates to working code, the class in Example 38-1 uses a property to trace access to an attribute named name; the actual stored data is named _name so it does not clash with the property.

class Person: 
    def __init__(self, name):
        self._name = name

    def getName(self):
        print('fetch...')
        return self._name

    def setName(self, value):
        print('change...')
        self._name = value

    def delName(self):
        print('remove...')
        del self._name

    name = property(getName, setName, delName, 'name property docs')

sue = Person('Sue Jones')           # sue has a managed attribute
print(sue.name)                     # Runs getName
sue.name = 'Susan Jones'            # Runs setName
print(sue.name)
del sue.name                        # Runs delName

print('-'*20)
bob = Person('Bob Smith')           # bob inherits property too
print(bob.name)
print(Person.name.__doc__)          # Or help(Person.name)

This particular property doesn’t do much—it simply intercepts and traces an attribute—but it serves to demonstrate the protocol. When this code is run, two instances inherit the property, just as they would any other attribute attached to their class. However, accesses to their name attribute are caught and managed by the code we provide:

$ python3 prop-person.py
fetch...
Sue Jones
change...
fetch...
Susan Jones
remove...
--------------------
fetch...
Bob Smith
name property docs

Like all class attributes, properties are inherited by both instances and lower subclasses. If we change our example as follows, for instance:

class Super:
    …the original Person class code…
    name = property(getName, setName, delName, 'name property docs')

class Person(Super):
    pass                            # Properties are inherited (class attrs)

sue = Person('Sue Jones')
…rest unchanged…

the output is the same—the Person subclass inherits the name property from Super, and the sue instance gets it from Person. In terms of inheritance, properties work the same as normal methods; because they have access to the self instance argument, they can access instance state information and methods irrespective of subclass depth, as the next section further demonstrates.

Computed Attributes

The example in the prior section simply traces attribute accesses. Usually, though, properties do much more—computing the value of an attribute dynamically when fetched, for instance, as Example 38-2 illustrates.

class PropSquare:
    def __init__(self, start):
        self.value = start

    def getX(self):                         # On attr fetch
        return self.value ** 2

    def setX(self, value):                  # On attr assign
        self.value = value

    X = property(getX, setX)                # No delete or docs

P = PropSquare(3)       # Two instances of class with property
Q = PropSquare(32)      # Each has different state information

print(P.X)              # 3 ** 2
P.X = 4
print(P.X)              # 4 ** 2
print(Q.X)              # 32 ** 2 (1024)

This class defines an attribute X that is accessed as though it were simple data, but really runs code to compute its value when fetched. The net effect triggers an implicit method call. When the code is run, the value is stored in the instance as state information, but each time we fetch it via the managed attribute, its value is automatically squared:

$ python3 prop-computed.py
9
16
1024

Notice that we’ve made two different instances—because property methods automatically receive a self argument, they have access to the state information stored in instances. In our case, this means the fetch computes the square of the subject instance’s own data.

Coding Properties with Decorators

Although we’re saving additional details until the next chapter, we introduced function decorator basics earlier, in Chapter 32. Recall that the function decorator syntax:

@decorator
def func(args): …

is automatically translated to this equivalent by Python to rebind the function name to the result of the decorator callable:

def func(args): …
func = decorator(func)

Because of this mapping, the property built-in can automatically serve as a decorator to define a function that will run automatically when an attribute is fetched:

class Person:
    @property
    def name(self): ...             # Rebinds: name = property(name)

When run, the decorated method is automatically passed to the first argument of the property built-in. This is really just alternative syntax for creating a property and rebinding the attribute name manually, but may be seen as more explicit in this role:

class Person:
    def name(self): …
    name = property(name)           # Manual equivalent to @property

class Person:
    def __init__(self, name):
        self._name = name

    @property
    def name(self):                 # name = property(name)
        'name property docs'
        print('fetch...')
        return self._name

    @name.setter
    def name(self, value):          # name = name.setter(name)
        print('change...')
        self._name = value

    @name.deleter
    def name(self):                 # name = name.deleter(name)
        print('remove...')
        del self._name

sue = Person('Sue Jones')           # sue has a managed attribute
print(sue.name)                     # Runs name getter (def name 1)
sue.name = 'Susan Jones'            # Runs name setter (def name 2)
print(sue.name)
del sue.name                        # Runs name deleter (def name 3)

print('-'*20)
bob = Person('Bob Smith')           # bob inherits property too
print(bob.name)
print(Person.name.__doc__)          # Or help(Person.name)

$ python3 prop-person-deco.py
fetch...
Sue Jones
change...
fetch...
Susan Jones
remove...
--------------------
fetch...
Bob Smith
name property docs

The Basics

As mentioned, descriptors are coded as separate classes and provide specially named accessor methods for the attribute access operations they wish to intercept—get, set, and deletion methods in the descriptor class are automatically run when the attribute assigned to the descriptor class instance is accessed in the corresponding way:

class Descriptor:
    "docstring goes here"
    def __get__(self, instance, owner): …        # Return attr value
    def __set__(self, instance, value): …        # Return nothing (None)
    def __delete__(self, instance): …            # Return nothing (None)

Classes with any of these methods are considered descriptors, and their methods are special when one of their instances is assigned to another class’s attribute—when the attribute is accessed, these methods are automatically invoked.

If any of these methods are absent, it generally means that the corresponding type of access is not supported. Unlike properties, however, omitting a __set__ allows the descriptor attribute’s name to be assigned and thus redefined in an instance, thereby hiding the descriptor—to make an attribute read-only, you must define __set__ to catch assignments and raise an exception.

Descriptors with __set__ methods also have some special-case implications for inheritance that we’ll largely defer until Chapter 40’s coverage of metaclasses and the complete inheritance specification. In short, a descriptor with a __set__ is known formally as a data descriptor and is given precedence over other names located by normal inheritance rules. The inherited descriptor for attribute __class__, for example, overrides the same name in an instance’s namespace dictionary. This also works to ensure that data descriptors you code in your own classes take precedence over others.

>>> class Descriptor:                  
        def __get__(self, instance, owner):
            print(self, instance, owner, sep='\n')

>>> class Subject:                           
        attr = Descriptor()            # Descriptor instance is class attr

>>> X = Subject()
>>> X.attr
<__main__.Descriptor object at 0x104bc9b20>
<__main__.Subject object at 0x104b8a570>
<class '__main__.Subject'> 

>>> Subject.attr
<__main__.Descriptor object at 0x104bc9b20>
None
<class '__main__.Subject'>

X.attr  =>  Descriptor.__get__(Subject.attr, X, Subject)

>>> class D:
        def __get__(*args): print('get')

>>> class C:
        a = D()                     # Attribute "a" is a descriptor instance

>>> X = C()
>>> X.a                             # Runs inherited descriptor __get__
get
>>> C.a
get
>>> X.a = 99                        # Stored on X, hiding C.a!
>>> X.a
99
>>> list(X.__dict__.keys())
['a']
>>> Y = C()
>>> Y.a                             # Y still inherits descriptor
get
>>> C.a
get

>>> class D:
        def __get__(*args): print('get')
        def __set__(*args): raise AttributeError('cannot set')

>>> class C:
        a = D()

>>> X = C()
>>> X.a                                 # Routed to C.a.__get__
get
>>> X.a = 99                            # Routed to C.a.__set__
AttributeError: cannot set

A First Example

To see how this all comes together in more realistic code, let’s get started with the same first example we wrote for properties. Example 38-4 defines a descriptor that intercepts access to an attribute named name in its clients. Its methods use their instance argument to access state information in the subject instance, where the name string is actually stored.

class Name:
    'name descriptor docs'

    def __get__(self, instance, owner):
        print('fetch...')
        return instance._name

    def __set__(self, instance, value):
        print('change...')
        instance._name = value

    def __delete__(self, instance):
        print('remove...')
        del instance._name

class Person:
    def __init__(self, name):
        self._name = name

    name = Name()                       # Assign descriptor to attr

sue = Person('Sue Jones')               # sue has a managed attribute
print(sue.name)                         # Runs Name.__get__
sue.name = 'Susan Jones'                # Runs Name.__set__
print(sue.name)
del sue.name                            # Runs Name.__delete__

print('-'*20)
bob = Person('Bob Smith')               # bob inherits descriptor too
print(bob.name)
print(Name.__doc__)                     # Or help(Name)

Notice in this code how we assign an instance of our descriptor class to a class attribute in the client class; because of this, it is inherited by all instances of the class, just like a class’s methods. Really, we must assign the descriptor to a class attribute like this—it won’t work if assigned to a self instance attribute instead. When the descriptor’s __get__ method is run, it is passed three objects to define its context:

• self is the Name class instance.

• instance is the Person class instance.

• owner is the Person class.

When this code is run the descriptor’s methods intercept accesses to the attribute, much like the property version. In fact, the output is the same again:

$ python3 desc-person.py
fetch...
Sue Jones
change...
fetch...
Susan Jones
remove...
--------------------
fetch...
Bob Smith
name descriptor docs

Also like in the property example, our descriptor class instance is a class attribute and thus is inherited by all instances of the client class and any subclasses. If we change the Person class in our example to the following, for instance, the output of our script is the same:

…
class Super:
    def __init__(self, name):
        self._name = name

    name = Name()

class Person(Super):                     # Descriptors are inherited (class attrs)
   pass
…

Also, note that when a descriptor class is not useful outside the client class, it’s perfectly reasonable to embed the descriptor’s definition inside its client syntactically. Here’s what our example looks like if we use a nested class:

class Person:
    def __init__(self, name):
        self._name = name

    class Name:                                 # Using a nested class
        'name descriptor docs'

        def __get__(self, instance, owner):
            …same…

        def __set__(self, instance, value):
            …same…

        def __delete__(self, instance):
            …same…

    name = Name()

When coded this way, Name becomes a local variable in the scope of the Person class statement, such that it won’t clash with any names outside the class. This version works the same as the original—we’ve simply moved the descriptor class definition into the client class’s scope—but the last line of the testing code must change to fetch the docstring from its new location (per unlisted file desc-person-nested.py in the example’s package):

…
print(Person.Name.__doc__)     # Differs: not Name.__doc__ outside class

Computed Attributes

As was the case when using properties, our first descriptor example of the prior section didn’t do much—it simply printed trace messages for attribute accesses as a demo. In practice, descriptors can also be used to compute attribute values each time they are fetched. Example 38-5 illustrates—it’s a rehash of the same example we coded for properties but uses a descriptor to automatically square an attribute’s value each time it is fetched.

class DescSquare:
    def __init__(self, start):                  # Each desc has own state
        self.value = start

    def __get__(self, instance, owner):         # On attr fetch
        return self.value ** 2

    def __set__(self, instance, value):         # On attr assign
        self.value = value                      # No delete or docs

class Client1:
    X = DescSquare(3)          # Assign descriptor instance to class attr

class Client2:
    X = DescSquare(32)         # Another instance in another client class
                               # Could also code two instances in same class
c1 = Client1()
c2 = Client2()

print(c1.X)                    # 3 ** 2
c1.X = 4
print(c1.X)                    # 4 ** 2
print(c2.X)                    # 32 ** 2 (1024)

When run, the output of this example is the same as that of the original property-based version, but here a descriptor class object is intercepting the attribute accesses instead of a property:

$ python3 desc-computed.py
9
16
1024

Using State Information in Descriptors

If you closely study the two descriptor examples we’ve written so far, you might notice that they get their information from different places—the first (the name attribute example) uses data stored on the client instance, and the second (the attribute squaring example) uses data attached to the descriptor object itself (a.k.a. self). In fact, descriptors can use both instance state and descriptor state, or any combination thereof:

• Descriptor state is used to manage either data internal to the workings of the descriptor or data that spans all instances. It can vary per attribute appearance (often per client class).

• Instance state records information related to and possibly created by the client class. It can vary per client-class instance (that is, per application object).

In other words, descriptor state is per-descriptor data, and instance state is per-client-instance data. As usual in OOP, you must choose state carefully. For example, you would not normally use descriptor state to record employee names since each client instance requires its own value—if stored in the descriptor, each client class instance will effectively share the same single copy. On the other hand, you would not usually use instance state to record data pertaining to descriptor implementation internals—if stored in each instance, there would be multiple varying copies.

Descriptor methods may use either state form, but descriptor state sometimes makes it unnecessary to use special naming conventions to avoid name collisions in the instance for data that is not instance specific. For example, the descriptor in Example 38-6 attaches information to its own instance, so it doesn’t clash with that on the client class’s instance—but also shares that information between two client instances.

class DescState:                           # Use descriptor state
    def __init__(self, value):
        self.value = value

    def __get__(self, instance, owner):    # On attr fetch
        print('DescState get')
        return self.value * 10

    def __set__(self, instance, value):    # On attr assign
        print('DescState set')
        self.value = value

# Client class
class CalcAttrs:
    X = DescState(2)                       # Descriptor class attr
    Y = 3                                  # Class attr
    def __init__(self):
        self.Z = 4                         # Instance attr

obj = CalcAttrs()
print(obj.X, obj.Y, obj.Z)                 # X is computed, others are not
obj.X = 5                                  # X assignment is intercepted
CalcAttrs.Y = 6                            # Y reassigned in class
obj.Z = 7                                  # Z assigned in instance
print(obj.X, obj.Y, obj.Z)

obj2 = CalcAttrs()                         # But X uses shared data, like Y!
print(obj2.X, obj2.Y, obj2.Z)

This code’s internal value information lives only in the descriptor, so there won’t be a collision if the same name is used in the client’s instance. Notice that only the descriptor attribute is managed here—get and set accesses to X are intercepted, but accesses to Y and Z are not (Y is attached to the client class and Z to the instance). When this code is run, X is computed when fetched, but its value is also the same for all client instances because it uses descriptor-level state:

$ python3 desc-state-desc.py
DescState get
20 3 4
DescState set
DescState get
50 6 7
DescState get
50 6 4

It’s also feasible for a descriptor to store or use an attribute attached to the client class’s instance instead of itself. Crucially, unlike data stored in the descriptor itself, this allows for data that can vary per client class instance. The descriptor in Example 38-7 assumes the instance has an attribute _X attached by the client class and uses it to compute the value of the attribute it represents.

class InstState:                           # Using instance state
    def __get__(self, instance, owner):
        print('InstState get')             # Assume set by client class
        return instance._X * 10

    def __set__(self, instance, value):
        print('InstState set')
        instance._X = value

# Client class
class CalcAttrs:
    X = InstState()                        # Descriptor class attr
    Y = 3                                  # Class attr
    def __init__(self):
        self._X = 2                        # Instance attr
        self.Z  = 4                        # Instance attr

obj = CalcAttrs()
print(obj.X, obj.Y, obj.Z)                 # X is computed, others are not
obj.X = 5                                  # X assignment is intercepted
CalcAttrs.Y = 6                            # Y reassigned in class
obj.Z = 7                                  # Z assigned in instance
print(obj.X, obj.Y, obj.Z)

obj2 = CalcAttrs()                         # But X differs now, like Z!
print(obj2.X, obj2.Y, obj2.Z)

Here, X is assigned to a descriptor as before that manages accesses. The new descriptor here, though, has no information itself, but it uses an attribute assumed to exist in the instance—that attribute is named _X, to avoid collisions with the name of the descriptor itself. When this version is run the results are similar, but the value of the descriptor attribute can vary per client instance due to the differing state policy:

$ python3 desc-state-inst.py
InstState get
20 3 4
InstState set
InstState get
50 6 7
InstState get
20 6 4

Both descriptor and instance state have roles. In fact, this is a general advantage that descriptors have over properties—because they have state of their own, they can easily retain data internally without adding it to the namespace of the client instance object. As a summary, the following uses both state sources—its self.data retains per-attribute information, while its instance.data can vary per client instance:

>>> class DescBoth:
        def __init__(self, data):
            self.data = data
        def __get__(self, instance, owner):
            return f'{self.data}, {instance.data}'
        def __set__(self, instance, value):
            instance.data = value

>>> class Client:
        def __init__(self, data):
            self.data = data
        managed = DescBoth('hack')

>>> I = Client('code')
>>> I.managed                      # Show both data sources
'hack, code' 
>>> I.managed = 'HACK'             # Change instance data
>>> I.managed
'hack, HACK'

We’ll revisit the implications of this choice in a case study later in this chapter. Before we move on, recall from Chapter 32’s coverage of slots that we can access “virtual” attributes like properties and descriptors with tools like dir and getattr, even though they don’t exist in the instance’s namespace dictionary. Whether you should access these this way probably varies per program—properties and descriptors may run arbitrary computation and may be less obviously instance “data” than slots:

>>> I.__dict__
{'data': 'HACK'}
>>> [x for x in dir(I) if not x.startswith('__')]
['data', 'managed'] 

>>> getattr(I, 'data')
'HACK'
>>> getattr(I, 'managed')
'hack, HACK'

>>> for attr in (x for x in dir(I) if not x.startswith('__')):
        print(f'{attr} => {getattr(I, attr)}')

data => HACK
managed => hack, HACK

The more generic __getattr__ and __getattribute__ tools we’ll explore soon are not designed to support this functionality: because they have no class-level attributes, their “virtual” attribute names do not appear in dir results (per Chapter 31, a __dir__ can provide a dir result, but it’s optional and uncommon). In exchange, they are also not limited to specific attribute names coded as properties or descriptors—tools that share even more than this behavior, as the next section explains.

How Properties and Descriptors Relate

As mentioned earlier, properties and descriptors are strongly related—the property built-in is just a convenient way to create a descriptor. Now that you know how both work, you should also be able to see that it’s possible to simulate the property built-in with a descriptor class, as demoed by Example 38-8.

class Property:
    def __init__(self, fget=None, fset=None, fdel=None, doc=None):
        self.fget = fget
        self.fset = fset
        self.fdel = fdel                                  # Save unbound methods
        self.__doc__ = doc                                # or other callables

    def __get__(self, instance, instancetype=None):
        if instance is None:
            return self
        if self.fget is None:
            raise AttributeError("can't get attribute")
        return self.fget(instance)                        # Pass instance to self
                                                          # in property accessors
    def __set__(self, instance, value):
        if self.fset is None:
            raise AttributeError("can't set attribute")
        self.fset(instance, value)

    def __delete__(self, instance):
        if self.fdel is None:
            raise AttributeError("can't delete attribute")
        self.fdel(instance)

class Person:
    def getName(self): 
        print('getName...')
    def setName(self, value): 
        print('setName...')
    name = Property(getName, setName)                     # Use like property()

x = Person()
x.name
x.name = 'Pat'
del x.name

This Property class catches attribute accesses with the descriptor protocol and routes requests to functions or methods passed in and saved in descriptor state when the class’s instance is created. Attribute fetches, for example, are routed from the Person class, to the Property class’s __get__ method, and back to the Person class’s getName. With descriptors, this “just works”:

$ python3 prop-desc-equiv.py
getName...
setName...
AttributeError: can't delete attribute

Note that this descriptor class equivalent only handles basic property usage, though; to use @ decorator syntax to also specify set and delete operations, we’d have to extend our Property class with setter and deleter methods, which would save the decorated accessor function and return the property object (self should suffice). Since the property built-in already does this, we’ll omit a formal coding of this extension here.

The Basics

In short, if a class defines or inherits the following methods, they will be run automatically when an instance is used in the operation described by the comments to the right:

def __getattr__(self, name):         # On undefined attribute fetch [obj.name]
def __getattribute__(self, name):    # On all attribute fetch [obj.name]
def __setattr__(self, name, value):  # On all attribute assignment [obj.name=value]
def __delattr__(self, name):         # On all attribute deletion [del obj.name]

In these, self is the subject instance object as usual, name is the string name of the attribute being accessed, and value is the object being assigned to the attribute. The two get methods normally return an attribute’s value, and the other two return nothing (None). All can raise exceptions to signal prohibited access.

For example, to catch every attribute fetch, we can use either of the first two previous methods, and to catch every attribute assignment we can use the third. The following uses __getattr__ for fetches:

class Catcher:
    def __getattr__(self, name):
        print('Get:', name)
    def __setattr__(self, name, value):
        print('Set:', name, value)

X = Catcher()
X.job                               # Prints "Get: job"
X.pay                               # Prints "Get: pay"
X.pay = 'bread'                     # Prints "Set: pay bread"

Using __getattribute__ works exactly the same in this specific case but has subtle looping potential which we’ll take up in the next section:

class Catcher:                               # On all attribute fetches
    def __getattribute__(self, name):        # Works same as getattr here
        print('Get:', name)                  # But prone to loops on general
    …rest unchanged…

Such a coding structure can be used to implement the delegation design pattern we met earlier in Chapter 31. Because all attributes are routed to interception methods generically, we can validate and pass them along to embedded, managed objects. As a refresher, the following class, borrowed from Chapter 31, traces every attribute fetch made to another object passed to the wrapper (proxy) class:

class Wrapper:
    def __init__(self, object):
        self.wrapped = object                    # Save object
    def __getattr__(self, attrname):
        print('Trace:', attrname)                # Trace fetch
        return getattr(self.wrapped, attrname)   # Delegate fetch

X = Wrapper([1, 2, 3])
X.append(4)                         # Prints "Trace: append"
print(X.wrapped)                    # Prints "[1, 2, 3, 4]"

There is no such analog for properties and descriptors, short of coding accessors for every attribute present in every wrapped object. On the other hand, when such generality is not required, generic accessor methods may incur additional calls for assignments in some contexts—a trade-off described in Chapter 30 and mentioned in the context of the case study example we’ll explore at the end of this chapter.

    def __getattribute__(self, name):
        x = self.other                                # LOOPS!

    def __getattribute__(self, name):
        x = object.__getattribute__(self, 'other')    # Force higher to avoid me

    def __setattr__(self, name, value):
        self.other = value                            # Recurs (and might LOOP!)

    def __setattr__(self, name, value):
        self.__dict__['other'] = value                # Use attr dict to avoid me

    def __setattr__(self, name, value):
        object.__setattr__(self, 'other', value)      # Force higher to avoid me

    def __getattribute__(self, name):
        x = self.__dict__['other']                    # Loops!

A First Example

Generic attribute management is not nearly as complicated as the prior section may have implied. To see how to put these ideas to work, Example 38-9 is the same first example we used for properties and descriptors in action again, this time implemented with attribute operator-overloading methods. Because these methods are so generic, we test attribute names here to know when a managed attribute is being accessed; others are allowed to pass normally.

class Person:
    def __init__(self, name):               # On [Person()]
        self._name = name                   # Triggers __setattr__!

    def __getattr__(self, attr):            # On [obj.undefined]
        print('get: ' + attr)
        if attr == 'name':                  # Intercept name: not stored
            return self._name               # Does not loop: real attr
        else:                               # Others are errors
            raise AttributeError(attr)

    def __setattr__(self, attr, value):     # On [obj.any = value]
        print('set: ' + attr)
        if attr == 'name':
            attr = '_name'                  # Set internal name
        self.__dict__[attr] = value         # Avoid looping here

    def __delattr__(self, attr):            # On [del obj.any]
        print('del: ' + attr)
        if attr == 'name':
            attr = '_name'                  # Avoid looping here too
        del self.__dict__[attr]             # but much less common

sue = Person('Sue Jones')           # sue has a managed attribute
print(sue.name)                     # Runs __getattr__
sue.name = 'Susan Jones'            # Runs __setattr__
print(sue.name)
del sue.name                        # Runs __delattr__

print('-'*20)
bob = Person('Bob Smith')           # bob's attrs work like sue's
print(bob.name)
#print(Person.name.__doc__)         # No direct equivalent here!

When this code is run, the same sort of output is produced, but this time it reflects our generic attribute-interception methods responding to Python’s normal operator-overloading mechanism:

$ python3 getattr-person.py
set: _name
get: name
Sue Jones
set: name
get: name
Susan Jones
del: name
--------------------
set: _name
get: name
Bob Smith

Notice how the attribute assignment in the __init__ constructor triggers __setattr__ too—this method catches every instance-attribute assignment, even those anywhere within the class itself, and those to underlying attributes like _name. Also note that, unlike with properties and descriptors, there’s no direct notion of specifying documentation for our attribute here; managed attributes exist within the code of our interception methods, not as distinct objects.

    # Replace just __getattr__ with this

    def __getattribute__(self, attr):                 # On [obj.any]
        print('get: ' + attr)
        if attr == 'name':                            # Intercept all names
            attr = '_name'                            # Map to internal name
        return object.__getattribute__(self, attr)    # Avoid looping here

$ python3 getattribute-person.py
set: _name
get: __dict__
get: name
Sue Jones
set: name
get: __dict__
get: name
Susan Jones
del: name
get: __dict__
--------------------
set: _name
get: __dict__
get: name
Bob Smith

Computed Attributes

As before, our prior example doesn’t really do anything but trace attribute fetches; it’s not much more work to compute an attribute’s value when fetched. As for properties and descriptors, Example 38-11 creates a virtual attribute X that runs a calculation when fetched.

class AttrSquare:
    def __init__(self, start):
        self.value = start                            # Triggers __setattr__!

    def __getattr__(self, attr):                      # On undefined attr fetch
        if attr == 'X':
            return self.value ** 2                    # value is not undefined
        else:
            raise AttributeError(attr)

    def __setattr__(self, attr, value):               # On all attr assignments
        if attr == 'X':
            attr = 'value'
        self.__dict__[attr] = value

A = AttrSquare(3)       # 2 instances of class with overloading
B = AttrSquare(32)      # Each has different state information

print(A.X)              # 3 ** 2
A.X = 4
print(A.X)              # 4 ** 2
print(B.X)              # 32 ** 2 (1024)

Running this code results in the same output that we got earlier when using properties and descriptors, but this script’s mechanics are based on generic attribute interception methods:

$ python3 getattr-computed.py
9
16
1024

class AttrSquare:
    def __init__(self, start):
        self.value = start                  # Triggers __setattr__!

    def __getattribute__(self, attr):       # On all attr fetches
        if attr == 'X':
            return self.value ** 2          # Triggers __getattribute__ again!
        else:
            return object.__getattribute__(self, attr)

    def __setattr__(self, attr, value):     # On all attr assignments
        if attr == 'X':
            attr = 'value'
        object.__setattr__(self, attr, value)

…self-test code same as Example 38-11…

    def __getattribute__(self, attr):
        if attr == 'X':
            return object.__getattribute__(self, 'value') ** 2

__getattr__ and __getattribute__ Compared

To summarize the coding differences between __getattr__ and __getattribute__, Example 38-13 uses both to implement three attributes—attr1 is a class attribute, attr2 is an instance attribute, and attr3 is a virtual managed attribute computed when fetched.

class GetAttr:
    attr1 = 1
    def __init__(self):
        self.attr2 = 2
    def __getattr__(self, attr):            # On undefined attrs only
        print('get:', attr)                 # Not on attr1: inherited from class
        if attr == 'attr3':                 # Not on attr2: stored on instance
            return 3
        else:
            raise AttributeError(attr)

X = GetAttr()
print(X.attr1)
print(X.attr2)
print(X.attr3)
print('-'*20)

class GetAttribute:
    attr1 = 1
    def __init__(self):
        self.attr2 = 2
    def __getattribute__(self, attr):       # On all attr fetches
        print('get:',  attr)                # Use superclass to avoid looping here
        if attr == 'attr3':
            return 3
        else:
            return object.__getattribute__(self, attr)

X = GetAttribute()
print(X.attr1)
print(X.attr2)
print(X.attr3)

When run, the __getattr__ version intercepts only attr3 accesses because it is undefined. The __getattribute__ version, on the other hand, intercepts all attribute fetches and must route those it does not manage to the superclass fetcher to avoid loops:

$ python3 getattr-v-getattribute.py
1
2
get: attr3
3
--------------------
get: attr1
1
get: attr2
2
get: attr3
3

Although __getattribute__ can catch more attribute fetches than __getattr__, in practice they are often just variations on a theme—if attributes are not physically stored, the two have the same effect.

Management Techniques Compared

To summarize the coding differences in all four attribute-management schemes we’ve just explored, let’s quickly step through a somewhat more comprehensive computed-attribute example using each technique. The first version, Example 38-14, uses properties to intercept and calculate attributes named square and cube. Notice how their base values are stored in names that begin with an underscore so they don’t clash with the names of the properties themselves.

"Two dynamically computed attributes with properties"

class Powers:
    def __init__(self, square, cube):
        self._square = square                      # _square is the base value
        self._cube   = cube                        # square is the property name

    def getSquare(self):
        return self._square ** 2
    def setSquare(self, value):
        self._square = value
    square = property(getSquare, setSquare)        # or @property decorator

    def getCube(self):
        return self._cube ** 3 
    cube = property(getCube)                       # Likewise

To do the same with descriptors, Example 38-15 defines the attributes with complete classes. Note that these descriptors store base values as instance state, so they must use leading underscores again so as not to clash with the names of descriptors; as called out by the final example of this chapter, we could avoid this renaming requirement by storing base values as descriptor state instead, but that doesn’t as directly address data that must vary per client-class instance.

"Same, but with descriptors (per-instance state)"

class DescSquare:
    def __get__(self, instance, owner):
        return instance._square ** 2
    def __set__(self, instance, value):
        instance._square = value

class DescCube:
    def __get__(self, instance, owner):
        return instance._cube ** 3

class Powers:
    square = DescSquare()
    cube   = DescCube()
    def __init__(self, square, cube):
        self._square = square                  # "self.square = square" works too,
        self._cube   = cube                    # because it triggers desc __set__!

To achieve the same result with __getattr__ fetch interception, Example 38-16 again stores base values with underscore-prefixed names so that accesses to managed names are undefined and thus invoke its method; it also needs to code a __setattr__ to intercept assignments and take care to avoid its potential for looping.

"Same, but with generic __getattr__ undefined-attribute interception"

class Powers:
    def __init__(self, square, cube):
        self._square = square
        self._cube   = cube

    def __getattr__(self, name):
        if name == 'square':
            return self._square ** 2
        elif name == 'cube':
            return self._cube ** 3
        else:
            raise TypeError('unknown attr:' + name)

    def __setattr__(self, name, value):
        if name == 'square':
            self.__dict__['_square'] = value             # Or use object
        else:
            self.__dict__[name] = value

The final option in Example 38-17, coding with __getattribute__, is similar to the prior version. Because it catches every attribute now, though, it must also route base value fetches to a superclass to avoid looping or extra calls—fetching self._square directly works too, but runs a second __getattribute__ call.

"Same, but with generic __getattribute__ all-attribute interception"

class Powers:
    def __init__(self, square, cube):
        self._square = square
        self._cube   = cube

    def __getattribute__(self, name):
        if name == 'square':
            return object.__getattribute__(self, '_square') ** 2
        elif name == 'cube':
            return object.__getattribute__(self, '_cube') ** 3
        else:
            return object.__getattribute__(self, name)

    def __setattr__(self, name, value):
        if name == 'square':
            object.__setattr__(self, '_square', value)   # Or use __dict__
        else:
            object.__setattr__(self, name , value)

To test, the following REPL session loops through a list of all four modules’ name strings and imports and fetches classes along the way. Each technique takes a different form in code, but all four produce the same result when run:

>>> from importlib import import_module
>>> mods = [f'all_four_{M}' for M in ('props', 'desc', 'getattr', 'getattribute')]
>>> for modname in mods:
        module = import_module(modname)    # Import by name string
        X = module.Powers(3, 4)            # This module's class (print to see)
        print(X.square)                    # 3 ** 2 = 9
        print(X.cube)                      # 4 ** 3 = 64
        X.square = 5
        print(X.square)                    # 5 ** 2 = 25
         
9
64
25
…repeated four times…

For more on how these alternatives compare, and other coding options, stay tuned for a more realistic application of them in the attribute-validation example ahead. First, though, we need to take a short side trip to study a pitfall associated with two of these tools—the generic attribute interceptors.

Intercepting Built-in Operation Attributes

If you’ve been reading this book linearly, some of this section is elaboration on earlier notes, especially Chapter 28’s “Delegating Built-Ins—or Not”. When __getattr__ and __getattribute__ were introduced here, it was stated that they intercept undefined- and all-attribute fetches, respectively, which makes them ideal for delegation-based coding patterns.

While this is true for both normally named and explicitly fetched attributes, their behavior needs some additional clarification. Specifically, the implicit method-name fetches of built-in operations will never automatically be routed to either of these two attribute-interceptor methods. This means that operator-overloading method calls cannot be delegated to wrapped objects unless wrapper classes somehow redefine these methods themselves.

For example, attribute fetches for the __str__, __add__, and __getitem__ methods un implicitly by printing, + expressions, and indexing, respectively, are not routed to either __getattr__ or __getattribute__. Instead, such methods are looked up in classes, and skip the instance and its attribute interceptors. Hence, there is no direct way to generically catch and delegate built-in operations like these.

This was a Python 3.X bifurcation, whose purported rationale involved metaclasses and optimization of built-in operations. Whatever its basis, all attributes—both __X__ and other—are still dispatched through the instance’s interceptor methods when accessed explicitly by name, so this qualifies as a glaring inconsistency: X.__add__ runs __getattr__, but X+Y which uses X.__add__ does not. The net effect complicates delegation-based code.

The good news is that wrapper classes can work around this constraint by redefining operator-overloading methods in the wrapper itself, in order to catch and delegate calls. These extra methods can be added either manually, with tools, or by definition in, and inheritance from, common superclasses. It’s more work for delegation classes when operator-overloading methods are part of a wrapped object’s interface, but it’s not a showstopper.

As a demo of the issue, consider the code in Example 38-18, which tests various attribute types and built-in operations on instances of classes containing __getattr__ and __getattribute__ methods.

class GetAttr:
    cattr = 88                       # Attrs stored on class and instance
    def __init__(self):              # These skip getattr, but not getattribute
       self.iattr = 77

    def __len__(self):               # Redefine for len(): doesn't run getattr
        print('__len__: 66')
        return 66

    def __getattr__(self, attr):     # Provide __str__ if asked, else dummy func
        print('getattr:', attr)      # Never run for __str__: inherited from object
        if attr == '__str__':
            return lambda *args: '[Getattr str]'
        else:
            return lambda *args: None

class GetAttribute:
    cattr = 88                       # Similar, but catch all attributes
    def __init__(self):              # Except implicit fetches for built-in ops
        self.iattr = 77               

    def __len__(self):               # Redefine for len(): doesn't run getattribute
        print('__len__: 66')         # But explicit fetches of inherited __str__ do
        return 66

    def __getattribute__(self, attr):
        print('getattribute:', attr)
        if attr == '__str__':
            return lambda *args: '[GetAttribute str]'
        else:
            return lambda *args: None

for Class in GetAttr, GetAttribute:
    print('\n' + Class.__name__.ljust(50, '='))
    X = Class()

    # Defined attributes trigger getattribute but not getattr

    X.cattr                   # Class attr    (defined – skips getattr)
    X.iattr                   # Instance attr (defined – skips getattr
    X.other                   # Missing attr
    len(X)                    # __len__ defined explicitly: moot

    # Built-in ops do not invoke either getattr or getattribute
    # No defaults are inherited for these from object superclass

    try:    X[0]              # Tries to invoke __getitem__
    except: print('fail []')
    try:    X + 99            # Ditto, __add__
    except: print('fail +')
    try:    X()               # Ditto, __call__
    except: print('fail ()')

    # But explicit calls invoke both catchers

    X.__getitem__(0)
    X.__add__(99)
    X.__call__()

    # The implied object superclass defines a __str__ that precludes getattr
    # But the absolute getattribute is not called for implicit fetches either
 
    print(X.__str__())        # __str__: explicit call => only __getattr__ skipped
    print(X)                  # __str__: implicit via built-in => both skipped

This file runs the same set of tests on each of its classes in turn. Match its following output with its tests and comments to see how it works. In short, neither __getattr__ nor __getattribute__ are run for any of the operator-overloading names invoked by built-in operations because such names are looked up in classes only:

$ python3 getattr-builtins.py

GetAttr===========================================
getattr: other
__len__: 66
fail []
fail +
fail ()
getattr: __getitem__
getattr: __add__
getattr: __call__
<__main__.GetAttr object at 0x10f76f020>
<__main__.GetAttr object at 0x10f76f020>

GetAttribute======================================
getattribute: cattr
getattribute: iattr
getattribute: other
__len__: 66
fail []
fail +
fail ()
getattribute: __getitem__
getattribute: __add__
getattribute: __call__
getattribute: __str__
[GetAttribute str]
<__main__.GetAttribute object at 0x10f74c440>

More generally, all explicit method-name attribute fetches are always routed to both attribute-interception methods, but none of the implicit operator-overloading methods trigger either attribute interception method when their attributes are fetched by built-in operations. Salient points in this demo worth calling out:

• __str__ access fails to be caught twice by __getattr__: once for the built-in print, and once for explicit fetches because a default is inherited from the built-in object implied above every topmost class.

• __str__ fails to be caught only once by the __getattribute__ catchall—during the built-in print operation. Explicit fetches bypass the inherited __str__ and run __getattribute__.

• __call__ fails to be caught in both schemes for built-in call expressions, but it is intercepted by both when fetched explicitly; unlike __str__, there is no inherited __call__ default in object to defeat __getattr__ in explicit fetches. The same goes for the __add__ of + operations.

• __len__ is handled by both classes because it is an explicitly defined method in the classes themselves—though its name is not routed to either __getattr__ or __getattribute__ if we delete the classes’ __len__ methods because the len built-in skips them as usual.

Again, the net effect is that operator-overloading methods implicitly run by built-in operations are never routed through either attribute interception method. Python begins the search for such attributes in classes and skips instance lookup mechanisms entirely. Normally, named attributes and explicit fetches start with the instance instead.

For a more realistic example of this phenomenon’s impact on delegation classes, stay tuned for the online-only “Decorators” chapter’s Private decorator—along with its coverage of multiple reusable workarounds.

Using Properties to Validate

Our first coding in Example 38-19 uses properties to manage three attributes. As usual, we could use simple methods instead of managed attributes, but properties help if we have already been using attributes in existing code. Properties run code automatically on attribute access but are focused on a specific set of attributes; they cannot be used to intercept all attributes generically.

To understand this code, it’s crucial to notice that the attribute assignments inside the __init__ constructor method trigger property setter methods too. When this method assigns to self.name, for example, it automatically invokes the setName method, which transforms the value and assigns it to an instance attribute called __name so it won’t clash with the property’s name.

This renaming, sometimes called name mangling, is important because properties use common instance state and have none of their own. Data is stored in an attribute called __name, and the attribute called name is always a property, not data. As we saw in Chapter 31, names like __name are known as pseudoprivate attributes and are changed by Python to include the enclosing class’s name when stored in the instance’s namespace; here, this helps keep the implementation-specific attributes distinct from others, including that of the property that manages them.

In the end, this class manages attributes called name, age, and acct; allows the attribute addr to be accessed directly; and provides a read-only attribute called remain that is entirely virtual and computed on demand. For comparison purposes, this property-based coding weighs in at 39 lines of code (including blank lines).

class CardHolder:
    acctlen = 8                                # Class data
    retireage = 62.5

    def __init__(self, acct, name, age, addr):
        self.acct = acct                       # Instance data
        self.name = name                       # These trigger prop setters too!
        self.age  = age                        # __X mangled to have class name
        self.addr = addr                       # addr is not managed
                                               # remain has no data
    def getName(self):
        return self.__name
    def setName(self, value):
        value = value.lower().replace(' ', '_')
        self.__name = value
    name = property(getName, setName)          # Or @ decorators for both

    def getAge(self):
        return self.__age
    def setAge(self, value):
        if value < 0 or value > 150:
            raise ValueError('invalid age')
        else:
            self.__age = value
    age = property(getAge, setAge)

    def getAcct(self):
        return self.__acct[:-3] + '***'
    def setAcct(self, value):
        value = value.replace('-', '')
        if len(value) != self.acctlen:
            raise TypeError('invalid acct number')
        else:
            self.__acct = value
    acct = property(getAcct, setAcct)

    def remainGet(self):                       # Could be a method, not attr
        return self.retireage - self.age       # Unless already using as attr
    remain = property(remainGet)

def loadclass():
    import sys, importlib
    modulename = sys.argv[1]                          # Module name in command line
    module = importlib.import_module(modulename)      # Import module by name string
    print(f'[Using: {module.CardHolder}]')            # No need for getattr() here
    return module.CardHolder

def printholder(who):
    print(who.acct, who.name, who.age, who.remain, who.addr, sep=' / ')

if __name__ == '__main__':
    CardHolder = loadclass()
    bob = CardHolder('1234-5678', 'Bob Smith', 40, '123 main st')
    printholder(bob)
    bob.name = 'Bob Q. Smith'
    bob.age  = 50
    bob.acct = '23-45-67-89'
    printholder(bob)

    sue = CardHolder('5678-12-34', 'Sue Jones', 35, '124 main st')
    printholder(sue)
    try:
        sue.age = 200
    except: print('Bad age for Sue')

    try:
        sue.remain = 5
    except: print("Can't set sue.remain")

    try:
        sue.acct = '1234567'
    except: print('Bad acct for Sue')

$ python3 validate_tester.py validate_properties
[Using: <class 'validate_properties.CardHolder'>]
12345*** / bob_smith / 40 / 22.5 / 123 main st
23456*** / bob_q._smith / 50 / 12.5 / 123 main st
56781*** / sue_jones / 35 / 27.5 / 124 main st
Bad age for Sue
Can't set sue.remain
Bad acct for Sue

Using Descriptors to Validate

Now, let’s recode our example using descriptors instead of properties. As we’ve seen, descriptors are very similar to properties in terms of functionality and roles; in fact, properties are basically a focused form of descriptor. Like properties, descriptors are designed to handle specific attributes, not generic attribute access. Unlike properties, descriptors can also have their own state, and so are perhaps a more general scheme.

class CardHolder:                                # Using shared descriptor state
    acctlen = 8                                  # Class data
    retireage = 62.5

    def __init__(self, acct, name, age, addr):
        self.acct = acct                         # Instance data
        self.name = name                         # These trigger __set__ calls too!
        self.age  = age                          # __X not needed: in descriptor
        self.addr = addr                         # addr is not managed
                                                 # remain has no data
    class Name:
        def __get__(self, instance, owner):      # Class names: CardHolder locals
            return self.name
        def __set__(self, instance, value):
            value = value.lower().replace(' ', '_')
            self.name = value
    name = Name()

    class Age:
        def __get__(self, instance, owner):
            return self.age                             # Use descriptor data
        def __set__(self, instance, value):
            if value < 0 or value > 150:
                raise ValueError('invalid age')
            else:
                self.age = value
    age = Age()

    class Acct:
        def __get__(self, instance, owner):
            return self.acct[:-3] + '***'
        def __set__(self, instance, value):
            value = value.replace('-', '')
            if len(value) != instance.acctlen:          # Use instance class data
                raise TypeError('invalid acct number')
            else:
                self.acct = value
    acct = Acct()

    class Remain:
        def __get__(self, instance, owner):
            return instance.retireage - instance.age    # Triggers Age.__get__
        def __set__(self, instance, value):
            raise TypeError('cannot set remain')        # Else set allowed here
    remain = Remain()

$ python3 validate_tester.py validate_descriptors1
[Using: <class 'validate_descriptors1.CardHolder'>]
…rest is same output as properties…

from validate_tester import loadclass
CardHolder = loadclass()

bob = CardHolder('1234-5678',  'Bob Smith', 40, '123 main st')
print('bob:', bob.name, bob.acct, bob.age, bob.addr)

sue = CardHolder('5678-12-34', 'Sue Jones', 35, '124 main st')
print('sue:', sue.name, sue.acct, sue.age, sue.addr)    # addr differs: client data
print('bob:', bob.name, bob.acct, bob.age, bob.addr)    # name,acct,age overwritten?

$ python3 validate_tester_plus.py validate_descriptors1
[Using: <class 'validate_descriptors1.CardHolder'>]
bob: bob_smith 12345*** 40 123 main st
sue: sue_jones 56781*** 35 124 main st
bob: sue_jones 56781*** 35 123 main st

class CardHolder:                                # Using per-client-instance state
    acctlen = 8                                  # Class data
    retireage = 62.5

    def __init__(self, acct, name, age, addr):
        self.acct = acct                         # Client instance data
        self.name = name                         # These trigger __set__ calls too!
        self.age  = age                          # __X needed: in client instance
        self.addr = addr                         # addr is not managed
                                                 # remain managed but has no data
    class Name:
        def __get__(self, instance, owner):      # Class names: CardHolder locals
            return instance.__name
        def __set__(self, instance, value):
            value = value.lower().replace(' ', '_')
            instance.__name = value
    name = Name()                                       # class.name vs mangled attr

    class Age:
        def __get__(self, instance, owner):
            return instance.__age                       # Use *instance* data
        def __set__(self, instance, value):
            if value < 0 or value > 150:
                raise ValueError('invalid age')
            else:
                instance.__age = value
    age = Age()                                         # class.age vs mangled attr

    class Acct:
        def __get__(self, instance, owner):
            return instance.__acct[:-3] + '***'
        def __set__(self, instance, value):
            value = value.replace('-', '')
            if len(value) != instance.acctlen:          # Use instance class data
                raise TypeError('invalid acct number')
            else:
                instance.__acct = value
    acct = Acct()                                       # class.acct vs mangled name

    class Remain:
        def __get__(self, instance, owner):
            return instance.retireage - instance.age    # Triggers Age.__get__
        def __set__(self, instance, value):
            raise TypeError('cannot set remain')        # Else set allowed here
    remain = Remain()

$ python3 validate_tester_plus.py validate_descriptors2
[Using: <class 'validate_descriptors2.CardHolder'>]
bob: bob_smith 12345*** 40 123 main st
sue: sue_jones 56781*** 35 124 main st
bob: bob_smith 12345*** 40 123 main st 

$ python3 validate_tester.py validate_descriptors2
…same output as properties, except class name…

>>> from validate_descriptors1 import CardHolder
>>> pat = CardHolder('1234-5678', 'Pat Smith', 40, '123 main st')
>>> pat.name
'pat_smith'
>>> CardHolder.name
'pat_smith'

>>> from validate_descriptors2 import CardHolder
>>> pat = CardHolder('1234-5678', 'Pat Smith', 40, '123 main st')
>>> pat.name
'pat_smith'
>>> CardHolder.name
AttributeError: 'NoneType' object has no attribute '_Name__name'

Using __getattr__ to Validate

As we’ve seen, the __getattr__ method intercepts all undefined attributes, so it can be more generic than using properties or descriptors. For our example, we simply test the attribute name to know when a managed attribute is being fetched; others are stored physically on the instance and so never reach __getattr__. Although this approach is more general than using properties or descriptors, extra work may be required to imitate the specific attribute focus of other tools. We need to check names at runtime—a multiple-choice that’s a prime role for the match statement—and we must code a __setattr__ in order to intercept and validate attribute assignments.

Example 38-24 hosts the __getattr__ version of our validations code. In the end, this class, like the prior two, manages attributes called name, age, and acct; allows the attribute addr to be accessed directly; and provides a read-only attribute called remain that is entirely virtual and is computed on demand.

As for the property and descriptor versions of this example, it’s critical to notice that the attribute assignments inside the __init__ constructor method trigger the class’s __setattr__ method too. When this method assigns to self.name, for example, it automatically invokes the __setattr__ method, which transforms the value and assigns it to an instance attribute called name. By storing name on the instance, it ensures that future accesses will not trigger __getattr__. In contrast, acct is stored as _acct, so that later accesses to acct do invoke __getattr__.

For comparison purposes, this alternative comes in at 34 lines of code—5 fewer than the property-based version and 11 fewer than the version using descriptors (though replacing if with match here added two lines, along with extra indentation). Clarity matters more than code size, of course, but extra code can imply extra development and maintenance work. Probably more important here are roles: generic tools like __getattr__ are better suited to generic delegation, while properties and descriptors are designed to manage specific attributes.

Also note again that the code here incurs extra calls when setting unmanaged attributes (e.g., addr), although no extra calls are incurred for fetching unmanaged attributes since they are defined. Though this will likely result in negligible overhead for most programs, the more narrowly focused properties and descriptors incur an extra call only when managed attributes are accessed, and also appear in dir results automatically when needed by generic tools.

class CardHolder:
    acctlen = 8                                  # Class data
    retireage = 62.5

    def __init__(self, acct, name, age, addr):
        self.acct = acct                         # Instance data
        self.name = name                         # These trigger __setattr__ too
        self.age  = age                          # _acct not mangled: name tested
        self.addr = addr                         # addr is not managed
                                                 # remain has no data
    def __getattr__(self, name):
        match name:
            case 'acct':                               # On undefined attr fetches
                return self._acct[:-3] + '***'         # name, age, addr are defined
            case 'remain':
                return self.retireage - self.age       # Doesn't trigger __getattr__
            case _:
                raise AttributeError(name)

    def __setattr__(self, name, value):
        match name:
            case 'name':                                 # On all attr assignments
                value = value.lower().replace(' ', '_')  # addr stored directly
            case 'age':                                  # acct mangled to _acct
                if value < 0 or value > 150:
                    raise ValueError('invalid age')
            case 'acct':
                name  = '_acct'
                value = value.replace('-', '')
                if len(value) != self.acctlen:
                    raise TypeError('invalid acct number')
            case 'remain':
                raise TypeError('cannot set remain')
        self.__dict__[name] = value                      # Avoid looping (or object)

When this code is run with either test script, it produces the same output (with a different class name):

$ python3 validate_tester.py validate_getattr
…same output as properties, except class name…

$ python3 validate_tester_plus.py validate_getattr
…same output as instance-state descriptors, except class name…

Using __getattribute__ to Validate

Our final variant uses the __getattribute__ catchall to intercept attribute fetches and manage them as needed. Every attribute fetch is caught here, so we test the attribute names to detect managed attributes and route all others to the superclass for normal fetch processing. This version uses the same __setattr__ to catch assignments as the prior version (there is no corresponding “__setattribute__” in Python—so far?).

Example 38-25 codes this last mod. It works very much like the __getattr__ version, so we won’t repeat the full description here. Note, though, that because every attribute fetch is routed to __getattribute__, we don’t need to mangle names to intercept them here (acct is stored as acct). On the other hand, this code must take care to route nonmanaged attribute fetches to a superclass to avoid looping or extra calls.

Also, notice that this version incurs extra calls for both setting and fetching unmanaged attributes (e.g., addr); if speed is paramount, this alternative may be the slowest of the bunch. For comparison purposes, this version amounts to 34 lines of code, just like the prior version (and again including two lines added by match).

class CardHolder:
    acctlen = 8                                  # Class data
    retireage = 62.5

    def __init__(self, acct, name, age, addr):
        self.acct = acct                         # Instance data
        self.name = name                         # These trigger __setattr__ too
        self.age  = age                          # acct not mangled: name tested
        self.addr = addr                         # addr is not managed
                                                 # remain has no data
    def __getattribute__(self, name):
        superget = object.__getattribute__                 # Don't loop: level up
        match name:
            case 'acct':                                   # On all attr fetches
                return superget(self, 'acct')[:-3] + '***'
            case 'remain':
                return superget(self, 'retireage') - superget(self, 'age')
            case _:
                return superget(self, name)                # name, age, addr: stored

    def __setattr__(self, name, value):
        match name:
            case 'name':                                   # On all attr assignments
                value = value.lower().replace(' ', '_')    # addr stored directly
            case 'age':
                if value < 0 or value > 150:
                    raise ValueError('invalid age')
            case 'acct':
                value = value.replace('-', '')
                if len(value) != self.acctlen:
                    raise TypeError('invalid acct number')
            case 'remain':
                raise TypeError('cannot set remain')
        self.__dict__[name] = value                         # Avoid loop, orig names

Both the __getattr__ and __getattribute__ scripts work the same as the property and per-client-instance descriptor versions when run by both tester scripts—four ways to achieve the same goal in Python, though they vary in structure and are perhaps less redundant in some other roles:

$ python3 validate_tester.py validate_getattribute
…same output as properties, except class name…

$ python3 validate_tester_plus.py validate_getattribute
…same output as instance-state descriptors, except class name…

Be sure to study and run this section’s code on your own for more pointers on managed-attribute coding techniques.

Test Your Knowledge: Answers

1. The __getattr__ method is run for explicit fetches of undefined attributes only (i.e., those not present on an instance and not inherited from any of its classes). By contrast, the __getattribute__ method is called for every explicit attribute fetch, whether the attribute is defined or not. Because of this, code inside a __getattr__ can freely fetch other attributes if they are defined, whereas __getattribute__ must use special code for all such attribute fetches to avoid looping or extra calls (it must route fetches to a superclass to skip itself). Neither method is run for implicit fetches of built-in operations (sans the next chapter’s heroics).

2. Properties serve a specific role, while descriptors are more general. Properties define get, set, and delete functions for a specific attribute; descriptors provide a class with methods for these actions, too, but they provide extra flexibility to support more arbitrary actions. In fact, properties are really a simple way to create a specific kind of descriptor—one that runs functions on attribute accesses. Coding differs too: a property is created with a built-in function, and a descriptor is coded with a class; thus, descriptors can leverage all the usual OOP features of classes, such as inheritance. Moreover, in addition to the instance’s state information, descriptors have local state of their own, which can sometimes avoid name collisions in the instance.

3. Properties can be coded with decorator syntax. Because the property built-in accepts a single function argument and returns a function, it can be used directly as a function decorator to define a fetch-access property. Due to the name rebinding behavior of decorators, the name of the decorated function is assigned to a property whose get accessor is set to the original function decorated (name=property(name)). Property setter and deleter attributes allow us to further add set and delete accessors with decoration syntax—they set the accessor to the decorated function and return the augmented property. Some may find this a bit clumsy, but this is subjective.

4. The __getattr__ and __getattribute__ methods are more generic: they can be used to catch arbitrarily many attributes. In contrast, each property or descriptor provides access interception for only one specific attribute—we can’t catch every attribute fetch with a single property or descriptor. On the other hand, properties and descriptors handle both attribute fetch and assignment by design: __getattr__ and __getattribute__ handle fetches only; to intercept assignments as well, __setattr__ must also be coded. The implementation is also different: __getattr__ and __getattribute__ are operator-overloading methods, whereas properties and descriptors are objects manually assigned to class attributes. Unlike the others, properties and descriptors can also sometimes avoid extra calls on assignment to unmanaged names and show up in dir results automatically, but are also narrower in scope—they can’t address generic delegation goals. In Python evolution, new features tend to offer alternatives but often do not fully subsume what came before.