The different data class builders have a lot in common, as summarized in Table 5-1.

Now let’s discuss those main features.

No Runtime Effect

Think about Python type hints as “documentation that can be verified by IDEs and type checkers.”

That’s because type hints have no impact on the runtime behavior of Python programs. Check out Example 5-9.

>>> import typing
>>> class Coordinate(typing.NamedTuple):
...     lat: float
...     lon: float
...
>>> trash = Coordinate('Ni!', None)
>>> print(trash)
Coordinate(lat='Ni!', lon=None)    

I told you: no type checking at runtime!

If you type the code of Example 5-9 in a Python module, it will run and display a meaningless Coordinate, with no error or warning:

$ python3 nocheck_demo.py
Coordinate(lat='Ni!', lon=None)

The type hints are intended primarily to support third-party type checkers, like Mypy or the PyCharm IDE built-in type checker. These are static analysis tools: they check Python source code “at rest,” not running code.

To see the effect of type hints, you must run one of those tools on your code—like a linter. For instance, here is what Mypy has to say about the previous example:

$ mypy nocheck_demo.py
nocheck_demo.py:8: error: Argument 1 to "Coordinate" has
incompatible type "str"; expected "float"
nocheck_demo.py:8: error: Argument 2 to "Coordinate" has
incompatible type "None"; expected "float"

As you can see, given the definition of Coordinate, Mypy knows that both arguments to create an instance must be of type float, but the assignment to trash uses a str and None.⁵

Now let’s talk about the syntax and meaning of type hints.

Variable Annotation Syntax

Both typing.NamedTuple and @dataclass use the syntax of variable annotations defined in PEP 526. This is a quick introduction to that syntax in the context defining attributes in class statements.

The basic syntax of variable annotation is:

var_name: some_type

The “Acceptable type hints” section in PEP 484 explains what are acceptable types, but in the context of defining a data class, these types are more likely to be useful:

• A concrete class, for example, str or FrenchDeck

• A parameterized collection type, like list[int], tuple[str, float], etc.

• typing.Optional, for example, Optional[str]—to declare a field that can be a str or None

You can also initialize the variable with a value. In a typing.NamedTuple or @dataclass declaration, that value will become the default for that attribute if the corresponding argument is omitted in the constructor call:

var_name: some_type = a_value

The Meaning of Variable Annotations

We saw in “No Runtime Effect” that type hints have no effect at runtime. But at import time—when a module is loaded—Python does read them to build the __annotations__ dictionary that typing.NamedTuple and @dataclass then use to enhance the class.

We’ll start this exploration with a simple class in Example 5-10, so that we can later see what extra features are added by typing.NamedTuple and @dataclass.

class DemoPlainClass:
    a: int           
    b: float = 1.1   
    c = 'spam'       

a becomes an entry in __annotations__, but is otherwise discarded: no attribute named a is created in the class.

b is saved as an annotation, and also becomes a class attribute with value 1.1.

c is just a plain old class attribute, not an annotation.

We can verify that in the console, first reading the __annotations__ of the DemoPlainClass, then trying to get its attributes named a, b, and c:

>>> from demo_plain import DemoPlainClass
>>> DemoPlainClass.__annotations__
{'a': <class 'int'>, 'b': <class 'float'>}
>>> DemoPlainClass.a
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: type object 'DemoPlainClass' has no attribute 'a'
>>> DemoPlainClass.b
1.1
>>> DemoPlainClass.c
'spam'

Note that the __annotations__ special attribute is created by the interpreter to record the type hints that appear in the source code—even in a plain class.

The a survives only as an annotation. It doesn’t become a class attribute because no value is bound to it.⁶ The b and c are stored as class attributes because they are bound to values.

None of those three attributes will be in a new instance of DemoPlainClass. If you create an object o = DemoPlainClass(), o.a will raise AttributeError, while o.b and o.c will retrieve the class attributes with values 1.1 and 'spam'—that’s just normal Python object behavior.

import typing

class DemoNTClass(typing.NamedTuple):
    a: int           
    b: float = 1.1   
    c = 'spam'       

>>> from demo_nt import DemoNTClass
>>> DemoNTClass.__annotations__
{'a': <class 'int'>, 'b': <class 'float'>}
>>> DemoNTClass.a
<_collections._tuplegetter object at 0x101f0f940>
>>> DemoNTClass.b
<_collections._tuplegetter object at 0x101f0f8b0>
>>> DemoNTClass.c
'spam'

>>> DemoNTClass.__doc__
'DemoNTClass(a, b)'

>>> nt = DemoNTClass(8)
>>> nt.a
8
>>> nt.b
1.1
>>> nt.c
'spam'

from dataclasses import dataclass

@dataclass
class DemoDataClass:
    a: int           
    b: float = 1.1   
    c = 'spam'       

>>> from demo_dc import DemoDataClass
>>> DemoDataClass.__annotations__
{'a': <class 'int'>, 'b': <class 'float'>}
>>> DemoDataClass.__doc__
'DemoDataClass(a: int, b: float = 1.1)'
>>> DemoDataClass.a
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: type object 'DemoDataClass' has no attribute 'a'
>>> DemoDataClass.b
1.1
>>> DemoDataClass.c
'spam'

>>> dc = DemoDataClass(9)
>>> dc.a
9
>>> dc.b
1.1
>>> dc.c
'spam'

>>> dc.a = 10
>>> dc.b = 'oops'

>>> dc.c = 'whatever'
>>> dc.z = 'secret stash'

Field Options

We’ve already seen the most basic field option: providing (or not) a default value with the type hint. The instance fields you declare will become parameters in the generated __init__. Python does not allow parameters without defaults after parameters with defaults, therefore after you declare a field with a default value, all remaining fields must also have default values.

Mutable default values are a common source of bugs for beginning Python developers. In function definitions, a mutable default value is easily corrupted when one invocation of the function mutates the default, changing the behavior of further invocations—an issue we’ll explore in “Mutable Types as Parameter Defaults: Bad Idea” (Chapter 6). Class attributes are often used as default attribute values for instances, including in data classes. And @dataclass uses the default values in the type hints to generate parameters with defaults for __init__. To prevent bugs, @dataclass rejects the class definition in Example 5-13.

@dataclass
class ClubMember:
    name: str
    guests: list = []

If you load the module with that ClubMember class, this is what you get:

$ python3 club_wrong.py
Traceback (most recent call last):
  File "club_wrong.py", line 4, in <module>
    class ClubMember:
  ...several lines omitted...
ValueError: mutable default <class 'list'> for field guests is not allowed:
use default_factory

The ValueError message explains the problem and suggests a solution: use default_factory. Example 5-14 shows how to correct ClubMember.

from dataclasses import dataclass, field

@dataclass
class ClubMember:
    name: str
    guests: list = field(default_factory=list)

In the guests field of Example 5-14, instead of a literal list, the default value is set by calling the dataclasses.field function with default_factory=list.

The default_factory parameter lets you provide a function, class, or any other callable, which will be invoked with zero arguments to build a default value each time an instance of the data class is created. This way, each instance of ClubMember will have its own list—instead of all instances sharing the same list from the class, which is rarely what we want and is often a bug.

If you browse the dataclasses module documentation, you’ll see a list field defined with a novel syntax, as in Example 5-15.

from dataclasses import dataclass, field

@dataclass
class ClubMember:
    name: str
    guests: list[str] = field(default_factory=list)  

list[str] means “a list of str.”

The new syntax list[str] is a parameterized generic type: since Python 3.9, the list built-in accepts that bracket notation to specify the type of the list items.

We’ll cover generics in Chapter 8. For now, note that Examples 5-14 and 5-15 are both correct, and the Mypy type checker does not complain about either of those class definitions.

The difference is that guests: list means that guests can be a list of objects of any kind, while guests: list[str] says that guests must be a list in which every item is a str. This will allow the type checker to find (some) bugs in code that puts invalid items in the list, or that read items from it.

The default_factory is likely to be the most common option of the field function, but there are several others, listed in Table 5-3.

The default option exists because the field call takes the place of the default value in the field annotation. If you want to create an athlete field with a default value of False, and also omit that field from the __repr__ method, you’d write this:

@dataclass
class ClubMember:
    name: str
    guests: list = field(default_factory=list)
    athlete: bool = field(default=False, repr=False)

The __init__ method generated by @dataclass only takes the arguments passed and assigns them—or their default values, if missing—to the instance attributes that are instance fields. But you may need to do more than that to initialize the instance. If that’s the case, you can provide a __post_init__ method. When that method exists, @dataclass will add code to the generated __init__ to call __post_init__ as the last step.

First, let’s look at the expected behavior of a ClubMember subclass named HackerClubMember, as described by doctests in Example 5-16.

"""
``HackerClubMember`` objects accept an optional ``handle`` argument::

    >>> anna = HackerClubMember('Anna Ravenscroft', handle='AnnaRaven')
    >>> anna
    HackerClubMember(name='Anna Ravenscroft', guests=[], handle='AnnaRaven')

If ``handle`` is omitted, it's set to the first part of the member's name::

    >>> leo = HackerClubMember('Leo Rochael')
    >>> leo
    HackerClubMember(name='Leo Rochael', guests=[], handle='Leo')

Members must have a unique handle. The following ``leo2`` will not be created,
because its ``handle`` would be 'Leo', which was taken by ``leo``::

    >>> leo2 = HackerClubMember('Leo DaVinci')
    Traceback (most recent call last):
      ...
    ValueError: handle 'Leo' already exists.

To fix, ``leo2`` must be created with an explicit ``handle``::

    >>> leo2 = HackerClubMember('Leo DaVinci', handle='Neo')
    >>> leo2
    HackerClubMember(name='Leo DaVinci', guests=[], handle='Neo')
"""

Note that we must provide handle as a keyword argument, because HackerClubMember inherits name and guests from ClubMember, and adds the handle field. The generated docstring for HackerClubMember shows the order of the fields in the constructor call:

>>> HackerClubMember.__doc__
"HackerClubMember(name: str, guests: list = <factory>, handle: str = '')"

Here, <factory> is a short way of saying that some callable will produce the default value for guests (in our case, the factory is the list class). The point is: to provide a handle but no guests, we must pass handle as a keyword argument.

The “Inheritance” section of the dataclasses module documentation explains how the order of the fields is computed when there are several levels of inheritance.

Example 5-17 shows the implementation.

from dataclasses import dataclass
from club import ClubMember

@dataclass
class HackerClubMember(ClubMember):                         
    all_handles = set()                                     
    handle: str = ''                                        

    def __post_init__(self):
        cls = self.__class__                                
        if self.handle == '':                               
            self.handle = self.name.split()[0]
        if self.handle in cls.all_handles:                  
            msg = f'handle {self.handle!r} already exists.'
            raise ValueError(msg)
        cls.all_handles.add(self.handle)                    

HackerClubMember extends ClubMember.

all_handles is a class attribute.

handle is an instance field of type str with an empty string as its default value; this makes it optional.

Get the class of the instance.

If self.handle is the empty string, set it to the first part of name.

If self.handle is in cls.all_handles, raise ValueError.

Add the new handle to cls.all_handles.

Example 5-17 works as intended, but is not satisfactory to a static type checker. Next, we’ll see why, and how to fix it.

If we type check Example 5-17 with Mypy, we are reprimanded:

$ mypy hackerclub.py
hackerclub.py:37: error: Need type annotation for "all_handles"
(hint: "all_handles: Set[<type>] = ...")
Found 1 error in 1 file (checked 1 source file)

Unfortunately, the hint provided by Mypy (version 0.910 as I review this) is not helpful in the context of @dataclass usage. First, it suggests using Set, but I am using Python 3.9 so I can use set—and avoid importing Set from typing. More importantly, if we add a type hint like set[…] to all_handles, @dataclass will find that annotation and make all_handles an instance field. We saw this happening in “Inspecting a class decorated with dataclass”.

The workaround defined in PEP 526—Syntax for Variable Annotations is ugly. To code a class variable with a type hint, we need to use a pseudotype named typing.ClassVar, which leverages the generics [] notation to set the type of the variable and also declare it a class attribute.

To make the type checker and @dataclass happy, this is how we are supposed to declare all_handles in Example 5-17:

    all_handles: ClassVar[set[str]] = set()

That type hint is saying:

all_handles is a class attribute of type set-of-str, with an empty set as its default value.

To code that annotation, we must import ClassVar from the typing module.

The @dataclass decorator doesn’t care about the types in the annotations, except in two cases, and this is one of them: if the type is ClassVar, an instance field will not be generated for that attribute.

The other case where the type of the field is relevant to @dataclass is when declaring init-only variables, our next topic.

Initialization Variables That Are Not Fields

Sometimes you may need to pass arguments to __init__ that are not instance fields. Such arguments are called init-only variables by the dataclasses documentation. To declare an argument like that, the dataclasses module provides the pseudotype InitVar, which uses the same syntax of typing.ClassVar. The example given in the documentation is a data class that has a field initialized from a database, and the database object must be passed to the constructor.

Example 5-18 shows the code that illustrates the “Init-only variables” section.

@dataclass
class C:
    i: int
    j: int = None
    database: InitVar[DatabaseType] = None

    def __post_init__(self, database):
        if self.j is None and database is not None:
            self.j = database.lookup('j')

c = C(10, database=my_database)

Note how the database attribute is declared. InitVar will prevent @dataclass from treating database as a regular field. It will not be set as an instance attribute, and the dataclasses.fields function will not list it. However, database will be one of the arguments that the generated __init__ will accept, and it will be also passed to __post_init__. If you write that method, you must add a corresponding argument to the method signature, as shown in Example 5-18.

This rather long overview of @dataclass covered the most useful features—some of them appeared in previous sections, like “Main Features” where we covered all three data class builders in parallel. The dataclasses documentation and PEP 526—Syntax for Variable Annotations have all the details.

In the next section, I present a longer example with @dataclass.

@dataclass Example: Dublin Core Resource Record

Often, classes built with @dataclass will have more fields than the very short examples presented so far. Dublin Core provides the foundation for a more typical @dataclass example.

The Dublin Core Schema is a small set of vocabulary terms that can be used to describe digital resources (video, images, web pages, etc.), as well as physical resources such as books or CDs, and objects like artworks.⁸

Dublin Core on Wikipedia

The standard defines 15 optional fields; the Resource class in Example 5-19 uses 8 of them.

from dataclasses import dataclass, field
from typing import Optional
from enum import Enum, auto
from datetime import date


class ResourceType(Enum):  
    BOOK = auto()
    EBOOK = auto()
    VIDEO = auto()


@dataclass
class Resource:
    """Media resource description."""
    identifier: str                                    
    title: str = '<untitled>'                          
    creators: list[str] = field(default_factory=list)
    date: Optional[date] = None                        
    type: ResourceType = ResourceType.BOOK             
    description: str = ''
    language: str = ''
    subjects: list[str] = field(default_factory=list)

This Enum will provide type-safe values for the Resource.type field.

identifier is the only required field.

title is the first field with a default. This forces all fields below to provide defaults.

The value of date can be a datetime.date instance, or None.

The type field default is ResourceType.BOOK.

Example 5-20 shows a doctest to demonstrate how a Resource record appears in code.

    >>> description = 'Improving the design of existing code'
    >>> book = Resource('978-0-13-475759-9', 'Refactoring, 2nd Edition',
    ...     ['Martin Fowler', 'Kent Beck'], date(2018, 11, 19),
    ...     ResourceType.BOOK, description, 'EN',
    ...     ['computer programming', 'OOP'])
    >>> book  # doctest: +NORMALIZE_WHITESPACE
    Resource(identifier='978-0-13-475759-9', title='Refactoring, 2nd Edition',
    creators=['Martin Fowler', 'Kent Beck'], date=datetime.date(2018, 11, 19),
    type=<ResourceType.BOOK: 1>, description='Improving the design of existing code',
    language='EN', subjects=['computer programming', 'OOP'])

The __repr__ generated by @dataclass is OK, but we can make it more readable. This is the format we want from repr(book):

    >>> book  # doctest: +NORMALIZE_WHITESPACE
    Resource(
        identifier = '978-0-13-475759-9',
        title = 'Refactoring, 2nd Edition',
        creators = ['Martin Fowler', 'Kent Beck'],
        date = datetime.date(2018, 11, 19),
        type = <ResourceType.BOOK: 1>,
        description = 'Improving the design of existing code',
        language = 'EN',
        subjects = ['computer programming', 'OOP'],
    )

Example 5-21 is the code of __repr__ to produce the format shown in the last snippet. This example uses dataclass.fields to get the names of the data class fields.

    def __repr__(self):
        cls = self.__class__
        cls_name = cls.__name__
        indent = ' ' * 4
        res = [f'{cls_name}(']                            
        for f in fields(cls):                             
            value = getattr(self, f.name)                 
            res.append(f'{indent}{f.name} = {value!r},')  

        res.append(')')                                   
        return '\n'.join(res)                             

Start the res list to build the output string with the class name and open parenthesis.

For each field f in the class…

…get the named attribute from the instance.

Append an indented line with the name of the field and repr(value)—that’s what the !r does.

Append closing parenthesis.

Build a multiline string from res and return it.

With this example inspired by the soul of Dublin, Ohio, we conclude our tour of Python’s data class builders.

Data classes are handy, but your project may suffer if you overuse them. The next section explains.

Data Class as Scaffolding

In this scenario, the data class is an initial, simplistic implementation of a class to jump-start a new project or module. With time, the class should get its own methods, instead of relying on methods of other classes to operate on its instances. Scaffolding is temporary; eventually your custom class may become fully independent from the builder you used to start it.

Python is also used for quick problem solving and experimentation, and then it’s OK to leave the scaffolding in place.

Data Class as Intermediate Representation

A data class can be useful to build records about to be exported to JSON or some other interchange format, or to hold data that was just imported, crossing some system boundary. Python’s data class builders all provide a method or function to convert an instance to a plain dict, and you can always invoke the constructor with a dict used as keyword arguments expanded with **. Such a dict is very close to a JSON record.

In this scenario, the data class instances should be handled as immutable objects—even if the fields are mutable, you should not change them while they are in this intermediate form. If you do, you’re losing the key benefit of having data and behavior close together. When importing/exporting requires changing values, you should implement your own builder methods instead of using the given “as dict” methods or standard constructors.

Now we change the subject to see how to write patterns that match instances of arbitrary classes, and not just the sequences and mappings we’ve seen in “Pattern Matching with Sequences” and “Pattern Matching with Mappings”.

Simple Class Patterns

We’ve already seen an example with simple class patterns used as subpatterns in “Pattern Matching with Sequences”:

        case [str(name), _, _, (float(lat), float(lon))]:

That pattern matches a four-item sequence where the first item must be an instance of str, and the last item must be a 2-tuple with two instances of float.

The syntax for class patterns looks like a constructor invocation. The following is a class pattern that matches float values without binding a variable (the case body can refer to x directly if needed):

    match x:
        case float():
            do_something_with(x)

But this is likely to be a bug in your code:

    match x:
        case float:  # DANGER!!!
            do_something_with(x)

In the preceding example, case float: matches any subject, because Python sees float as a variable, which is then bound to the subject.

The simple pattern syntax of float(x) is a special case that applies only to nine blessed built-in types, listed at the end of the “Class Patterns” section of PEP 634—Structural Pattern Matching: Specification:

bytes   dict   float   frozenset   int   list   set   str   tuple

In those classes, the variable that looks like a constructor argument—e.g., the x in float(x)—is bound to the whole subject instance or the part of the subject that matches a subpattern, as exemplified by str(name) in the sequence pattern we saw earlier:

        case [str(name), _, _, (float(lat), float(lon))]:

If the class is not one of those nine blessed built-ins, then the argument-like variables represent patterns to be matched against attributes of an instance of that class.

Keyword Class Patterns

To understand how to use keyword class patterns, consider the following City class and five instances in Example 5-22.

import typing

class City(typing.NamedTuple):
    continent: str
    name: str
    country: str


cities = [
    City('Asia', 'Tokyo', 'JP'),
    City('Asia', 'Delhi', 'IN'),
    City('North America', 'Mexico City', 'MX'),
    City('North America', 'New York', 'US'),
    City('South America', 'São Paulo', 'BR'),
]

Given those definitions, the following function would return a list of Asian cities:

def match_asian_cities():
    results = []
    for city in cities:
        match city:
            case City(continent='Asia'):
                results.append(city)
    return results

The pattern City(continent='Asia') matches any City instance where the continent attribute value is equal to 'Asia', regardless of the values of the other attributes.

If you want to collect the value of the country attribute, you could write:

def match_asian_countries():
    results = []
    for city in cities:
        match city:
            case City(continent='Asia', country=cc):
                results.append(cc)
    return results

The pattern City(continent='Asia', country=cc) matches the same Asian cities as before, but now the cc variable is bound to the country attribute of the instance. This also works if the pattern variable is called country as well:

        match city:
            case City(continent='Asia', country=country):
                results.append(country)

Keyword class patterns are very readable, and work with any class that has public instance attributes, but they are somewhat verbose.

Positional class patterns are more convenient in some cases, but they require explicit support by the class of the subject, as we’ll see next.

Positional Class Patterns

Given the definitions from Example 5-22, the following function would return a list of Asian cities, using a positional class pattern:

def match_asian_cities_pos():
    results = []
    for city in cities:
        match city:
            case City('Asia'):
                results.append(city)
    return results

The pattern City('Asia') matches any City instance where the first attribute value is 'Asia', regardless of the values of the other attributes.

If you want to collect the value of the country attribute, you could write:

def match_asian_countries_pos():
    results = []
    for city in cities:
        match city:
            case City('Asia', _, country):
                results.append(country)
    return results

The pattern City('Asia', _, country) matches the same cities as before, but now the country variable is bound to the third attribute of the instance.

I’ve mentioned “first” or “third” attribute, but what does that really mean?

What makes City or any class work with positional patterns is the presence of a special class attribute named __match_args__, which the class builders in this chapter automatically create. This is the value of __match_args__ in the City class:

>>> City.__match_args__
('continent', 'name', 'country')

As you can see, __match_args__ declares the names of the attributes in the order they will be used in positional patterns.

In “Supporting Positional Pattern Matching” we’ll write code to define __match_args__ for a class we’ll create without the help of a class builder.

Time for a chapter summary.