Class Objects Provide Default Behavior

When we run a class statement, we get a class object. Here’s a rundown of the main properties of Python classes:

• The class statement creates a class object and assigns it a name. Just like the function def statement, the Python class is an executable statement. When reached and run, it generates a new class object and assigns it to the first name in the class header. Also, like defs, class statements typically run when the files they are coded in are first imported or run as a top-level script.

• Assignments inside class statements make class attributes. Just like in module files, top-level assignments within a class statement (not nested in a def) generate attributes in a class object. Technically, the class statement defines a local scope that morphs into the attribute namespace of the class object, just like a module’s global scope. After running a class statement, class attributes may be accessed by name qualification: class.name.

• Class attributes provide object state and behavior. Attributes of a class object record state information and behavior to be shared by all instances created from the class. Most notably and commonly, function def statements nested inside a class generate methods, which process instances.

Instance Objects Are Concrete Items

When we call a class object, we get an instance object. Here’s an overview of the key points behind class instances:

• Calling a class object like a function makes a new instance object. Each time a class is called, it creates and returns a new instance object. Instances represent material items in your program’s domain.

• Each instance object inherits class attributes and gets its own namespace. Instance objects created from classes are new namespaces; they start out empty but inherit attributes that live in the class objects from which they were generated.

• Assignments to attributes of self in methods make per-instance attributes. Inside a class’s method functions, the first argument (called self by convention) references the instance object being processed; assignments to attributes of self create or change data in the instance, not the class.

The end result is that classes define common, shared data and behavior, and generate instances. Instances reflect palpable application entities and record per-instance data that may vary per object.

A First Example

Let’s turn to a real example to show how these ideas work in practice. To begin, let’s define a class named FirstClass by running a Python class statement interactively in any REPL:

>>> class FirstClass:               # Define a class object
        def setdata(self, value):   # Define class's methods
            self.data = value       # self is the instance
        def display(self):
            print(self.data)        # self.data: per instance

We’re working interactively here, but typically, such a statement would be run when the module file it is coded in is imported. Like functions created with defs, this class won’t even exist until Python reaches and runs this statement.

Like all compound statements, the class starts with a header line that lists the class name, followed by a body of one or more nested and (usually) indented statements. Here, the nested statements are defs; they define functions that implement the behavior the class means to export.

As we learned in Part IV, def is really an assignment. Here, it assigns function objects to the names setdata and display in the class statement’s scope, and so generates attributes attached to the class—FirstClass.setdata and FirstClass.display. In fact, any name assigned at the top level of the class’s nested block becomes an attribute of the class.

Functions inside a class are usually and traditionally called methods. They’re coded with normal defs, and they support everything we’ve learned about functions already—they can have defaults, return values, yield items on request, and so on. But in a method function, the first argument automatically receives an implied instance object when called—the subject of the call. Let’s create a couple of instances of our class to see how this works:

>>> x = FirstClass()                # Make two instances
>>> y = FirstClass()                # Each is a new namespace

By calling the class this way (notice the parentheses), we generate instance objects, which are just namespaces that have access to their classes’ attributes. Properly speaking, at this point, we have three objects: two instances and a class. And really, we have three linked namespaces, as sketched in Figure 27-1. In OOP terms, we say that x “is a” FirstClass, as is y—they both inherit names attached to the class.

Figure 27-1. Classes and instances: namespaces in a class tree searched by inheritance

The two instances start out empty but have links back to the class from which they were generated. If we qualify an instance with the name of an attribute that lives in the class object, Python fetches the name from the class by inheritance search (unless it also lives in the instance):

>>> x.setdata('coding')             # Call methods: self is x
>>> y.setdata(3.14159)              # Runs: FirstClass.setdata(y, 3.14159)

Neither x nor y has a setdata attribute of its own, so to find it, Python follows the link from instance to class. And that’s about all there is to inheritance in Python: it happens at attribute qualification time, and it just involves looking up names in linked objects—here, by following the is-a links in Figure 27-1.

In the setdata function inside FirstClass, the value passed in is assigned to self.data. Within a method, self—the name given to the leftmost argument by convention—automatically refers to the instance being processed (x or y at this point), so the assignments store values in the instances’ namespaces, not the class’s. That’s how the data names in Figure 27-1 are created.

Because classes can generate multiple instances, methods must go through the self argument to get to the instance to be processed. When we call the class’s display method to print self.data, we see that it’s different in each instance; on the other hand, the name display itself is the same in x and y, as it comes (is inherited) from the class:

>>> x.display()                     # Runs: FirstClass.display(x) 
coding
>>> y.display()                     # self.data differs in each instance
3.14159

Notice that we stored different object types in the data member in each instance—a string and a floating-point number. As with everything else in Python, instance attributes (sometimes called members) are not predeclared and have no type constraints; they spring into existence the first time they are assigned values, just like simple variables. In fact, if we were to call display on one of our instances before calling setdata, we would trigger an undefined name error—the attribute named data doesn’t even exist in memory until it is assigned within the setdata method.

As another way to appreciate how dynamic this model is, consider that we can change instance attributes either inside the class itself, by assigning to self in methods, or outside the class, by assigning to an explicit instance object:

>>> x.data = 'hacking'              # Can get/set attributes
>>> x.display()                     # Outside the class too
hacking

Although less common, we could even generate an entirely new attribute in the instance’s namespace by assigning to its name outside the class’s method functions:

>>> x.anothername = 'apps'          # Can set new attributes here too!

This would attach a new attribute called anothername, which may or may not be used by any of the class’s methods, to the instance object x. Classes usually create all of the instance’s attributes by assignment to the self argument, but they don’t have to—programs can fetch, change, or create attributes on any objects to which they have references.

It usually doesn’t make sense to add data that the class cannot use, and it’s possible to prevent this with extra “privacy” code based on attribute-access operator overloading, as we’ll discuss elsewhere in this book (in Chapter 30 and the online-only “Decorators” chapter). Still, free attribute access translates to less syntax, and there are cases where it’s even useful—for example, in coding data records of the sort we’ll build later in this chapter.

A Second Example

To illustrate the role of inheritance, this next example builds on the previous one. First, we’ll define a new class, SecondClass, that inherits all of FirstClass’s names and provides one of its own:

>>> class SecondClass(FirstClass):                   # Inherits setdata
        def display(self):                           # Changes display
            print(f'Current value = "{self.data}"')

SecondClass defines the display method to print with a different format. By defining an attribute with the same name as an attribute in FirstClass, SecondClass effectively replaces the display attribute in its superclass.

Recall that inheritance searches proceed upward from instances to subclasses to superclasses, stopping at the first appearance of the attribute name that it finds. In this case, since the display name in SecondClass will be found before the one in FirstClass, we say that SecondClass overrides FirstClass’s display. Sometimes we call this act of replacing attributes by redefining them lower in the tree overloading.

The net effect here is that SecondClass specializes FirstClass by changing the behavior of the display method. On the other hand, SecondClass (and any instances created from it) still inherits the setdata method in FirstClass verbatim. Let’s make a new instance to demonstrate:

>>> z = SecondClass()
>>> z.setdata('LP6e')       # Finds setdata in FirstClass
>>> z.display()             # Finds overridden method in SecondClass
Current value = "LP6e"

As before, we make a SecondClass instance object by calling it. The setdata call still runs the version in FirstClass, but this time the display attribute comes from SecondClass and prints a custom message. Figure 27-2 sketches the namespaces involved.

Figure 27-2. Specialization: overriding inherited names by redefining them in subclasses

Now, here’s a crucial thing to notice about OOP: the specialization introduced in SecondClass is completely external to FirstClass. That is, it doesn’t affect existing or future FirstClass objects, like the x from the prior example (assuming we’re continuing the same REPL session):

>>> x.display()             # x is still a FirstClass instance (old message)
hacking

Rather than changing FirstClass, we customized it. Naturally, this is an artificial example, but as a rule, because inheritance allows us to make changes like this in external components (i.e., in subclasses), classes often support extension and reuse better than functions or modules can.

Classes Are Attributes in Modules

Before we move on, remember that there’s nothing magic about a class name. It’s just a variable assigned to an object when the class statement runs, and the object can be referenced with any normal expression. For instance, if our FirstClass were coded in a module file instead of being typed interactively, we could import it and use its name normally in a class header line:

from modulename import FirstClass           # Copy name into my scope
class SecondClass(FirstClass):              # Use class name directly
    def display(self): …

Or equivalently:

import modulename                           # Access the whole module
class SecondClass(modulename.FirstClass):   # Qualify to reference
    def display(self): …

Like everything else, class names always live within a module, so they must follow all the rules we studied in Part V. For example, more than one class can be coded in a single module file—like other statements in a module, class statements are run during imports to define names, and these names become distinct module attributes. More generally, each module may arbitrarily mix any number of variables, functions, and classes, and all names in a module behave the same way. The following hypothetical file demonstrates:

# names.py
var1 = 6                 # names.var1
var2 = 3.12             
def func1(): …           # names.func1
def func2(): …               
class Cls1: …            # names.Cls1
class Cls2: …            # names.Cls2

This holds true even if the module and class happen to have the same name. For example, given the following imaginary file, person.py:

class person: …

we need to go through the module to fetch the class as usual:

import person                                 # Import module
x = person.person()                           # Class within module

Although this path may look redundant, it’s required: person.person refers to the person class inside the person module. Saying just person gets the module, not the class, unless the from statement is used:

from person import person                     # Get class from module
x = person()                                  # Use class name

As with any other variable, we can never see a class in a file without first importing and somehow fetching it from its enclosing file. If this seems confusing, don’t use the same name for a module and a class within it. In fact, common convention in Python recommends that class names should begin with an uppercase letter, and module names with a lowercase letter, to help make them more distinct (it’s not required, but nearly common enough to be a rule):

import person                                 # Lowercase for modules
x = person.Person()                           # Uppercase for classes

Also, keep in mind that although classes and modules are both namespaces for attaching attributes, they correspond to very different source code structures: a module reflects an entire file, but a class is a statement within a file. We’ll say more about such distinctions later in this part of the book.

A Third Example

On to another example. This time, we’ll define a subclass of the prior section’s SecondClass that implements three specially named attributes that Python will call automatically:

• __init__ is run when a new instance object is created: self is the new ThirdClass object.¹

• __add__ is run when a ThirdClass instance appears in a + expression.

• __str__ is run when an object is printed (technically, when it’s converted to its print string by the str built-in function or its Python-internals equivalent).

Our new subclass also defines a normally named method called mul, which changes the instance object in place. Here’s the new subclass (copy-and-pasters: in REPLs, you may need to omit blank lines added here for clarity):

>>> class ThirdClass(SecondClass):                     # Inherit from SecondClass
        def __init__(self, value):                     # On "ThirdClass(value)"
            self.data = value

        def __add__(self, other):                      # On "self + other"
            return ThirdClass(self.data + other)

        def __str__(self):                             # On "print(self)", "str()"
            return f'[ThirdClass: {self.data}]'

        def mul(self, other):                          # In-place change: named
            self.data *= other

ThirdClass “is a” SecondClass, so its instances inherit the customized display method from SecondClass of the preceding section. This time, though, ThirdClass creation calls pass an object to the value argument in the __init__ constructor, where it is assigned to self.data. The net effect is that ThirdClass arranges to set the data attribute automatically at construction time, instead of requiring later setdata calls:

>>> a = ThirdClass(3)               # __init__ called
>>> a.display()                     # Inherited method called
Current value = "3"

ThirdClass objects can also now show up in + expressions and print calls. For +, Python passes the instance object on the left to the self argument in __add__ and the value on the right to other, as illustrated in Figure 27-3; whatever __add__ returns becomes the result of the + expression (more on its result in a moment):

>>> b = a + 3                       # __add__: makes a new instance
>>> b.display()                     # b has all ThirdClass methods
Current value = "6"

For print, Python passes the object being printed to self in __str__; whatever string this method returns is taken to be the print string for the object. With __str__ (or its broader twin __repr__, which we’ll leverage in the next chapter), we can use a normal print to display objects of this class, instead of calling the display method. As a contrast, the added mul method also changes the instance in place for a named call:

>>> print(b)                        # __str__: returns display string
[ThirdClass: 6]
>>> a.mul(3)                        # mul: changes instance in place
>>> print(a)                        # Print this instance's data
[ThirdClass: 9]

Figure 27-3. Operator overloading: class methods are run for operators and operations

Specially named methods such as __init__, __add__, and __str__ are inherited by subclasses and instances, just like any other names assigned in a class. If they’re not coded in a class, Python looks for such names in all its superclasses, as usual. As for all attributes, the lowest (most specific) version is used.

Operator-overloading method names are also not built-in or reserved words; they are just attributes that Python looks for when objects appear in various contexts. Python usually calls them automatically, but they may occasionally be called by your code as well. For example, the __init__ method is often called manually to trigger required initialization steps in a superclass constructor, as you’ll see in the next chapter.

Returning results—or not

Some operator-overloading methods like __str__ require results, but others are more flexible. For example, notice how the __add__ method makes and returns a new instance object of its class, by calling ThirdClass with the result value—which in turn triggers __init__ to initialize the result. This is a common convention, and explains why b in the listing has a display method; it’s a ThirdClass object too, because that’s what + returns for this class’s objects. This essentially propagates the object type.

By contrast, mul changes the current instance object in place, by reassigning the self attribute. We could overload the * expression to do the same with __mul__, but this would be too different from the behavior of * for built-in types such as numbers and strings, for which it always makes new objects. Per common practice, overloaded operators should work the same way that built-in operations do. Because operator overloading is really just an expression-to-method dispatch mechanism, though, you can interpret operators any way you like in your own classes. Also, stay tuned for Chapter 30’s related coverage of in-place operator methods (preview: __imul__ handles *=).

Other operator-overloading methods

Although we won’t cover every operator-overloading method in this book, we’ll survey additional common operator-overloading techniques in Chapter 30. Again, while this is an optional tool that doesn’t apply to most application programs, __str__ is not uncommon, and the __init__ constructor method is a norm that will be present in most Python classes you’ll come across. In fact, even though instance attributes need not be predeclared in Python, you can usually find out which attributes an instance will have by inspecting its class’s __init__ method.

Classes: Under the Hood

In fact, as we’ll explore in more detail in Chapter 29, the attributes of a namespace object are usually implemented as dictionaries, and class inheritance trees are, generally speaking, just dictionaries with links to other dictionaries. If you know where to look, you can see this explicitly.

For example, the __dict__ attribute is the namespace dictionary for most class-based objects, much as in modules. Some classes may also—or instead—define attributes in __slots__, an advanced and seldom-used feature called slots with impacts that we’ll note along the way, but whose full coverage we’ll largely postpone until Chapter 32. Normally, though, __dict__ literally is the attribute namespace of an instance or class.

To illustrate, the following inspects the namespaces of objects in the prior section’s REPL session, filtering out built-ins in the class with a generator expression as we’ve done before leaving just the names we’ve assigned:

>>> list(key for key in rec.__dict__ if not key.startswith('__'))
['name', 'age'] 

>>> list(x.__dict__)
['name']
>>> list(y.__dict__)
[]

Here, the class’s namespace dictionary shows the name and age attributes we assigned to it, x has its own name, and y is still empty. Because of this model, an attribute can often be fetched by either dictionary indexing or attribute notation, but only if it’s present on the object in question—attribute notation kicks off inheritance search, but indexing looks in the single object only. As you’ll see later, both have valid roles, and the dir built-in collects inherited names:

>>> x.name, x.__dict__['name']        # Attributes present here are dict keys
('Sue', 'Sue')

>>> x.age                             # But attribute fetch checks classes too
40
>>> x.__dict__['age']                 # Indexing dict does not do inheritance
KeyError: 'age'
>>> x.__dict__                        # Object namespace versus inheritance
{'name': 'Sue'}
>>> [attr for attr in dir(x) if attr[:2] != '__']
['age', 'name']

In addition, to facilitate inheritance search on attribute fetches, each instance has a link to its class that Python creates for us—it’s called __class__, if you want to inspect it:

>>> x.__class__                       # Instance-to-class link
<class '__main__.rec'>

Classes also have a __bases__ attribute, which is a tuple of references to a class’s superclass objects—in this example just the implicit and automatic object root class we’ll explore later:

>>> rec.__bases__                     # Class-to-superclasses link
(<class 'object'>,)

These two attributes are how class trees are literally represented in memory by Python. Internal details like these are not required knowledge—class trees are implied by the code you run, and their search is normally automatic—but they can often help demystify the model.

The main point here is that Python’s class model is extremely dynamic. Classes and instances are just linked namespace objects, with attributes created on the fly by assignment. Those assignments usually happen within the class statements you code, but they can occur anywhere you have a reference to one of the objects in the tree.

In fact, even methods, normally created by a def nested in a class, can be created completely independently of any class object. The following, for example, defines a simple function outside of any class that takes one argument:

>>> def uppername(obj):
        return obj.name.upper()       # Still needs a self argument (obj)

There is nothing about a class here yet—it’s a simple function, and it can be called as such at this point, provided we pass in an object obj with a name attribute, whose value in turn has an upper method—our class and instances happen to fit the expected interface and kick off string uppercase conversion:

>>> uppername(rec), uppername(x), uppername(y)
('PAT', 'SUE', 'PAT')

If we assign this simple function to an attribute of our class, though, it becomes a method, callable through any instance, as well as through the class name itself as long as we pass in an instance manually—a technique we’ll leverage further in the next chapter:²

>>> rec.method = uppername            # Now it's a class's method!

>>> x.method()                        # Run method to process x
'SUE'
>>> y.method()                        # Same, but pass y to self
'PAT'
>>> rec.method(x)                     # Can call through instance or class
'SUE'

Normally, classes are filled out by class statements, and instance attributes are created by assignments to self attributes in method functions. The point again, though, is that they don’t have to be; OOP in Python really is mostly about looking up attributes in linked namespace objects.

Records Revisited: Classes Versus Dictionaries

Although the simple classes of the prior section are meant to illustrate class model basics, the techniques they employ can also be used for real work. For example, Chapters 8 and 9 showed how to use dictionaries, tuples, and lists to record properties of entities in our programs, generically called records. It turns out that classes can often serve better in this role—they package information like dictionaries, but can also bundle processing logic in the form of methods. For reference, here is an example for tuple- and dictionary-based records we used earlier in the book (using one of many dictionary coding techniques):

>>> rec = ('Pat', 40.5, ['dev', 'mgr'])     # Tuple-based record
>>> print(rec[0])
Pat

>>> rec = {}
>>> rec['name'] = 'Pat'                     # Dictionary-based record
>>> rec['age']  = 40.5                      # Or {...}, dict(n=v), etc.
>>> rec['jobs'] = ['dev', 'mgr']
>>> 
>>> print(rec['name'])
Pat

As we just saw, though, there are also multiple ways to do the same with classes. Perhaps the simplest is this—trading keys for attributes:

>>> class rec: pass

>>> rec.name = 'Pat'                        # Class-based record
>>> rec.age  = 40.5
>>> rec.jobs = ['dev', 'mgr']
>>> 
>>> print(rec.name)
Pat

This code has substantially less syntax than the dictionary equivalent. It uses an empty class statement to generate an empty namespace object, which we then fill out by assigning class attributes over time, as before. This works, but a new class statement will be required for each distinct record we will need. Perhaps more typically, we can instead generate instances of an empty class to represent each distinct entity:

>>> class rec: pass

>>> pers1 = rec()                           # Instance-based records
>>> pers1.name = 'Bob'
>>> pers1.jobs = ['dev', 'mgr']
>>> pers1.age  = 40.5
>>>
>>> pers2 = rec()
>>> pers2.name = 'Sue'
>>> pers2.jobs = ['dev', 'cto']
>>>
>>> pers1.name, pers2.name
('Bob', 'Sue')

Here, we make two records from the same class. Instances start out life empty, just like classes. We then fill in the records by assigning to attributes. This time, though, there are two separate objects, and hence two separate name attributes. In fact, instances of the same class don’t even have to have the same set of attribute names; in this example, one has a unique age name. Instances really are distinct namespaces, so each has a distinct attribute dictionary. Although they are normally filled out consistently by a class’s methods, they are more flexible than you might expect.

Finally, we might instead code a more full-blown class to implement the record and its processing—something that data-oriented dictionaries do not directly support:

>>> class Person:
        def __init__(self, name, jobs, age=None):      # class = data + logic
            self.name = name
            self.jobs = jobs
            self.age  = age
        def info(self):
            return (self.name, self.jobs)

>>> rec1 = Person('Bob', ['dev', 'mgr'], 40.5)         # Construction calls
>>> rec2 = Person('Sue', ['dev', 'cto'])
>>>
>>> rec1.jobs, rec2.info()                             # Attributes + methods
(['dev', 'mgr'], ('Sue', ['dev', 'cto']))

This scheme also makes multiple instances, but the class is not empty this time: we’ve added logic (methods) to initialize instances at construction time and collect attributes into a tuple on request. The constructor imposes some consistency on instances here by always setting the name, job, and age attributes, even though the latter can be omitted when an object is made. Together, the class’s methods and instance attributes create a package, which combines both data and logic.

We could further extend this code by adding logic to compute salaries, parse names, and so on. Ultimately, we might link the class into a larger hierarchy to inherit and customize an existing set of methods via the automatic attribute search of classes, or perhaps even store instances of the class in a file with Python object pickling to make them persistent. In fact, we will—in the next chapter, we’ll expand on this analogy between classes and records with a more realistic running example that demonstrates class basics in action.

To be fair to other tools, in this form, the two class construction calls above more closely resemble dictionaries made all at once, but still seem less cluttered and provide extra processing methods. In fact, the class’s construction calls more closely resemble Chapter 9’s named tuples—which makes sense, given that named tuples really are classes with extra logic to map attributes to tuple offsets:

>>> rec = dict(name='Pat', jobs=['dev', 'mgr'], age=40.5)        # Dictionaries

>>> rec = {'name': 'Pat', 'jobs': ['dev', 'mgr'], 'age': 40.5}   # Ditto

>>> …setup code…
>>> rec = Rec('Pat', ['dev', 'mgr'], 40.5)                       # Named tuples

In the end, although types like dictionaries and tuples are flexible, classes allow us to add behavior to objects in ways that built-in types and simple functions do not directly support. Although we can store functions in dictionaries too (e.g., under key info to mimic the class’s method), using them to process implied instances is nowhere near as natural and structured as it is in classes, and key lookup has no notion of inheritance to enable customization.

But to vet the purported benefits of classes firsthand, you’ll have to move ahead to the next chapter.