Starting with Mypy

To begin type checking, I run the mypy command on the messages.py module:

…/no_hints/ $ pip install mypy
[lots of messages omitted...]
…/no_hints/ $ mypy messages.py
Success: no issues found in 1 source file

Mypy with default settings finds no problem with Example 8-1.

If a function signature has no annotations, Mypy ignores it by default—unless configured otherwise.

For Example 8-2, I also have pytest unit tests. This is the code in messages_test.py.

from pytest import mark

from messages import show_count

@mark.parametrize('qty, expected', [
    (1, '1 part'),
    (2, '2 parts'),
])
def test_show_count(qty, expected):
    got = show_count(qty, 'part')
    assert got == expected

def test_show_count_zero():
    got = show_count(0, 'part')
    assert got == 'no parts'

Now let’s add type hints, guided by Mypy.

Making Mypy More Strict

The command-line option --disallow-untyped-defs makes Mypy flag any function definition that does not have type hints for all its parameters and for its return value.

Using --disallow-untyped-defs on the test file produces three errors and a note:

…/no_hints/ $ mypy --disallow-untyped-defs messages_test.py
messages.py:14: error: Function is missing a type annotation
messages_test.py:10: error: Function is missing a type annotation
messages_test.py:15: error: Function is missing a return type annotation
messages_test.py:15: note: Use "-> None" if function does not return a value
Found 3 errors in 2 files (checked 1 source file)

For the first steps with gradual typing, I prefer to use another option: --disallow-incomplete-defs. Initially, it tells me nothing:

…/no_hints/ $ mypy --disallow-incomplete-defs messages_test.py
Success: no issues found in 1 source file

Now I can add just the return type to show_count in messages.py:

def show_count(count, word) -> str:

This is enough to make Mypy look at it. Using the same command line as before to check messages_test.py will lead Mypy to look at messages.py again:

…/no_hints/ $ mypy --disallow-incomplete-defs messages_test.py
messages.py:14: error: Function is missing a type annotation
for one or more arguments
Found 1 error in 1 file (checked 1 source file)

Now I can gradually add type hints function by function, without getting warnings about functions that I haven’t annotated. This is a fully annotated signature that satisfies Mypy:

def show_count(count: int, word: str) -> str:

[mypy]
python_version = 3.9
warn_unused_configs = True
disallow_incomplete_defs = True

A Default Parameter Value

The show_count function in Example 8-1 only works with regular nouns. If the plural can’t be spelled by appending an 's', we should let the user provide the plural form, like this:

>>> show_count(3, 'mouse', 'mice')
'3 mice'

Let’s do a little “type-driven development.” First we add a test that uses that third argument. Don’t forget to add the return type hint to the test function, otherwise Mypy will not check it.

def test_irregular() -> None:
    got = show_count(2, 'child', 'children')
    assert got == '2 children'

Mypy detects the error:

…/hints_2/ $ mypy messages_test.py
messages_test.py:22: error: Too many arguments for "show_count"
Found 1 error in 1 file (checked 1 source file)

Now I edit show_count, adding the optional plural parameter in Example 8-3.

def show_count(count: int, singular: str, plural: str = '') -> str:
    if count == 1:
        return f'1 {singular}'
    count_str = str(count) if count else 'no'
    if not plural:
        plural = singular + 's'
    return f'{count_str} {plural}'

Now Mypy reports “Success.”

def hex2rgb(color=str) -> tuple[int, int, int]:

colors.py:24: error: Function is missing a type
    annotation for one or more arguments

The following details are considered good style for type hints:

• No space between the parameter name and the :; one space after the :

• Spaces on both sides of the = that precedes a default parameter value

On the other hand, PEP 8 says there should be no spaces around the = if there is no type hint for that particular parameter.

Using None as a Default

In Example 8-3, the parameter plural is annotated as str, and the default value is '', so there is no type conflict.

I like that solution, but in other contexts None is a better default. If the optional parameter expects a mutable type, then None is the only sensible default—as we saw in “Mutable Types as Parameter Defaults: Bad Idea”.

To have None as the default for the plural parameter, here is what the signature would look like:

from typing import Optional

def show_count(count: int, singular: str, plural: Optional[str] = None) -> str:

Let’s unpack that:

• Optional[str] means plural may be a str or None.

• You must explicitly provide the default value = None.

If you don’t assign a default value to plural, the Python runtime will treat it as a required parameter. Remember: at runtime, type hints are ignored.

Note that we need to import Optional from the typing module. When importing types, it’s good practice to use the syntax from typing import X to reduce the length of the function signatures.

Now that we’ve had a first practical view of gradual typing, let’s consider what the concept of type means in practice.

The keystone of any gradual type system is the Any type, also known as the dynamic type. When a type checker sees an untyped function like this:

def double(x):
    return x * 2

def double(x: Any) -> Any:
    return x * 2

def double(x: object) -> object:

def double(x: object) -> object:
    return x * 2

…/birds/ $ mypy double_object.py
double_object.py:2: error: Unsupported operand types for * ("object" and "int")
Found 1 error in 1 file (checked 1 source file)

Of course, no type can support every possible operation, so using Any prevents the type checker from fulfilling its core mission: detecting potentially illegal operations before your program crashes with a runtime exception.

class T1:
    ...

class T2(T1):
    ...

def f1(p: T1) -> None:
    ...

o2 = T2()

f1(o2)  # OK

def f2(p: T2) -> None:
    ...

o1 = T1()

f2(o1)  # type error

def f3(p: Any) -> None:
    ...

o0 = object()
o1 = T1()
o2 = T2()

f3(o0)  #
f3(o1)  #  all OK: rule #2
f3(o2)  #

def f4():  # implicit return type: `Any`
    ...

o4 = f4()  # inferred type: `Any`

f1(o4)  #
f2(o4)  #  all OK: rule #3
f3(o4)  #

Simple types like int, float, str, and bytes may be used directly in type hints. Concrete classes from the standard library, external packages, or user defined—FrenchDeck, Vector2d, and Duck—may also be used in type hints.

Abstract base classes are also useful in type hints. We’ll get back to them as we study collection types, and in “Abstract Base Classes”.

Among classes, consistent-with is defined like subtype-of: a subclass is consistent-with all its superclasses.

However, “practicality beats purity,” so there is an important exception, which I discuss in the following tip.

Optional and Union Types

We saw the Optional special type in “Using None as a Default”. It solves the problem of having None as a default, as in this example from that section:

from typing import Optional

def show_count(count: int, singular: str, plural: Optional[str] = None) -> str:

The construct Optional[str] is actually a shortcut for Union[str, None], which means the type of plural may be str or None.

plural: Optional[str] = None    # before
plural: str | None = None       # after

The ord built-in function’s signature is a simple example of Union—it accepts str or bytes, and returns an int:⁸

def ord(c: Union[str, bytes]) -> int: ...

Here is an example of a function that takes a str, but may return a str or a float:

from typing import Union

def parse_token(token: str) -> Union[str, float]:
    try:
        return float(token)
    except ValueError:
        return token

If possible, avoid creating functions that return Union types, as they put an extra burden on the user—forcing them to check the type of the returned value at runtime to know what to do with it. But the parse_token in the preceding code is a reasonable use case in the context of a simple expression evaluator.

Union[] requires at least two types. Nested Union types have the same effect as a flattened Union. So this type hint:

Union[A, B, Union[C, D, E]]

is the same as:

Union[A, B, C, D, E]

Union is more useful with types that are not consistent among themselves. For example: Union[int, float] is redundant because int is consistent-with float. If you just use float to annotate the parameter, it will accept int values as well.

Generic Collections

Most Python collections are heterogeneous. For example, you can put any mixture of different types in a list. However, in practice that’s not very useful: if you put objects in a collection, you are likely to want to operate on them later, and usually this means they must share at least one common method.9

Generic types can be declared with type parameters to specify the type of the items they can handle.

For example, a list can be parameterized to constrain the type of the elements in it, as you can see in Example 8-8.

def tokenize(text: str) -> list[str]:
    return text.upper().split()

In Python ≥ 3.9, it means that tokenize returns a list where every item is of type str.

The annotations stuff: list and stuff: list[Any] mean the same thing: stuff is a list of objects of any type.

PEP 585—Type Hinting Generics In Standard Collections lists collections from the standard library accepting generic type hints. The following list shows only those collections that use the simplest form of generic type hint, container[item]:

list        collections.deque        abc.Sequence   abc.MutableSequence
set         abc.Container            abc.Set        abc.MutableSet
frozenset   abc.Collection

The tuple and mapping types support more complex type hints, as we’ll see in their respective sections.

As of Python 3.10, there is no good way to annotate array.array, taking into account the typecode constructor argument, which determines whether integers or floats are stored in the array. An even harder problem is how to type check integer ranges to prevent OverflowError at runtime when adding elements to arrays. For example, an array with typecode='B' can only hold int values from 0 to 255. Currently, Python’s static type system is not up to this challenge.

Now let’s see how to annotate generic tuples.

Tuple Types

There are three ways to annotate tuple types:

>>> shanghai = 31.2304, 121.4737
>>> geohash(shanghai)
'wtw3sjq6q'

from geolib import geohash as gh  # type: ignore  

PRECISION = 9

def geohash(lat_lon: tuple[float, float]) -> str:  
    return gh.encode(*lat_lon, PRECISION)

from typing import NamedTuple

from geolib import geohash as gh  # type: ignore

PRECISION = 9

class Coordinate(NamedTuple):
    lat: float
    lon: float

def geohash(lat_lon: Coordinate) -> str:
    return gh.encode(*lat_lon, PRECISION)

def display(lat_lon: tuple[float, float]) -> str:
    lat, lon = lat_lon
    ns = 'N' if lat >= 0 else 'S'
    ew = 'E' if lon >= 0 else 'W'
    return f'{abs(lat):0.1f}°{ns}, {abs(lon):0.1f}°{ew}'

>>> animals = 'drake fawn heron ibex koala lynx tahr xerus yak zapus'.split()
>>> table = columnize(animals)
>>> table
[('drake', 'koala', 'yak'), ('fawn', 'lynx', 'zapus'), ('heron', 'tahr'),
 ('ibex', 'xerus')]
>>> for row in table:
...     print(''.join(f'{word:10}' for word in row))
...
drake     koala     yak
fawn      lynx      zapus
heron     tahr
ibex      xerus

list[tuple[str, ...]]

from collections.abc import Sequence

def columnize(
    sequence: Sequence[str], num_columns: int = 0
) -> list[tuple[str, ...]]:
    if num_columns == 0:
        num_columns = round(len(sequence) ** 0.5)
    num_rows, reminder = divmod(len(sequence), num_columns)
    num_rows += bool(reminder)
    return [tuple(sequence[i::num_rows]) for i in range(num_rows)]

Generic mapping types are annotated as MappingType[KeyType, ValueType]. The built-in dict and the mapping types in collections and collections.abc accept that notation in Python ≥ 3.9. For earlier versions, you must use typing.Dict and other mapping types from the typing module, as described in “Legacy Support and Deprecated Collection Types”.

Example 8-14 shows a practical use of a function returning an inverted index to search Unicode characters by name—a variation of Example 4-21 more suitable for server-side code that we’ll study in Chapter 21.

Given starting and ending Unicode character codes, name_index returns a dict[str, set[str]], which is an inverted index mapping each word to a set of characters that have that word in their names. For example, after indexing ASCII characters from 32 to 64, here are the sets of characters mapped to the words 'SIGN' and 'DIGIT', and how to find the character named 'DIGIT EIGHT':

>>> index = name_index(32, 65)
>>> index['SIGN']
{'$', '>', '=', '+', '<', '%', '#'}
>>> index['DIGIT']
{'8', '5', '6', '2', '3', '0', '1', '4', '7', '9'}
>>> index['DIGIT'] & index['EIGHT']
{'8'}

Example 8-14 shows the source code for charindex.py with the name_index function. Besides a dict[] type hint, this example has three features appearing for the first time in the book.

import sys
import re
import unicodedata
from collections.abc import Iterator

RE_WORD = re.compile(r'\w+')
STOP_CODE = sys.maxunicode + 1

def tokenize(text: str) -> Iterator[str]:  
    """return iterable of uppercased words"""
    for match in RE_WORD.finditer(text):
        yield match.group().upper()

def name_index(start: int = 32, end: int = STOP_CODE) -> dict[str, set[str]]:
    index: dict[str, set[str]] = {}  
    for char in (chr(i) for i in range(start, end)):
        if name := unicodedata.name(char, ''):  
            for word in tokenize(name):
                index.setdefault(word, set()).add(char)
    return index

tokenize is a generator function. Chapter 17 is about generators.

The local variable index is annotated. Without the hint, Mypy says: Need type annotation for 'index' (hint: "index: dict[<type>, <type>] = ...").

I used the walrus operator := in the if condition. It assigns the result of the unicodedata.name() call to name, and the whole expression evaluates to that result. When the result is '', that’s falsy, and the index is not updated.¹¹

Abstract Base Classes

Be conservative in what you send, be liberal in what you accept.

Postel’s law, a.k.a. the Robustness Principle

Table 8-1 lists several abstract classes from collections.abc. Ideally, a function should accept arguments of those abstract types—or their typing equivalents before Python 3.9—and not concrete types. This gives more flexibility to the caller.

from collections.abc import Mapping

def name2hex(name: str, color_map: Mapping[str, int]) -> str:

def name2hex(name: str, color_map: dict[str, int]) -> str:

Now color_map must be a dict or one of its subtypes, such as defaultDict or OrderedDict. In particular, a subclass of collections.UserDict would not pass the type check for color_map, despite being the recommended way to create user-defined mappings, as we saw in “Subclassing UserDict Instead of dict”. Mypy would reject a UserDict or an instance of a class derived from it, because UserDict is not a subclass of dict; they are siblings. Both are subclasses of abc.MutableMapping.¹²

Therefore, in general it’s better to use abc.Mapping or abc.MutableMapping in parameter type hints, instead of dict (or typing.Dict in legacy code). If the name2hex function doesn’t need to mutate the given color_map, the most accurate type hint for color_map is abc.Mapping. That way, the caller doesn’t need to provide an object that implements methods like setdefault, pop, and update, which are part of the MutableMapping interface, but not of Mapping. This has to do with the second part of Postel’s law: “Be liberal in what you accept.”

Postel’s law also tells us to be conservative in what we send. The return value of a function is always a concrete object, so the return type hint should be a concrete type, as in the example from “Generic Collections”—which uses list[str]:

def tokenize(text: str) -> list[str]:
    return text.upper().split()

Under the entry of typing.List, the Python documentation says:

Generic version of list. Useful for annotating return types. To annotate arguments it is preferred to use an abstract collection type such as Sequence or Iterable.

A similar comment appears in the entries for typing.Dict and typing.Set.

Remember that most ABCs from collections.abc and other concrete classes from collections, as well as built-in collections, support generic type hint notation like collections.deque[str] starting with Python 3.9. The corresponding typing collections are only needed to support code written in Python 3.8 or earlier. The full list of classes that became generic appears in the “Implementation” section of PEP 585—Type Hinting Generics In Standard Collections.

To wrap up our discussion of ABCs in type hints, we need to talk about the numbers ABCs.

The typing.List documentation I just quoted recommends Sequence and Iterable for function parameter type hints.

One example of the Iterable argument appears in the math.fsum function from the standard library:

def fsum(__seq: Iterable[float]) -> float:

Example 8-15 is another example using an Iterable parameter that produces items that are tuple[str, str]. Here is how the function is used:

>>> l33t = [('a', '4'), ('e', '3'), ('i', '1'), ('o', '0')]
>>> text = 'mad skilled noob powned leet'
>>> from replacer import zip_replace
>>> zip_replace(text, l33t)
'm4d sk1ll3d n00b p0wn3d l33t'

Example 8-15 shows how it’s implemented.

from collections.abc import Iterable

FromTo = tuple[str, str]  

def zip_replace(text: str, changes: Iterable[FromTo]) -> str:  
    for from_, to in changes:
        text = text.replace(from_, to)
    return text

FromTo is a type alias: I assigned tuple[str, str] to FromTo, to make the signature of zip_replace more readable.

changes needs to be an Iterable[FromTo]; that’s the same as Iterable[tuple[str, str]], but shorter and easier to read.

from typing import TypeAlias

FromTo: TypeAlias = tuple[str, str]

A parameterized generic is a generic type, written as list[T], where T is a type variable that will be bound to a specific type with each usage. This allows a parameter type to be reflected on the result type.

Example 8-16 defines sample, a function that takes two arguments: a Sequence of elements of type T, and an int. It returns a list of elements of the same type T, picked at random from the first argument.

Example 8-16 shows the implementation.

from collections.abc import Sequence
from random import shuffle
from typing import TypeVar

T = TypeVar('T')

def sample(population: Sequence[T], size: int) -> list[T]:
    if size < 1:
        raise ValueError('size must be >= 1')
    result = list(population)
    shuffle(result)
    return result[:size]

Here are two examples of why I used a type variable in sample:

• If called with a tuple of type tuple[int, ...]—which is consistent-with Sequence[int]—then the type parameter is int, so the return type is list[int].

• If called with a str—which is consistent-with Sequence[str]—then the type parameter is str, so the return type is list[str].

Another example is the statistics.mode function from the standard library, which returns the most common data point from a series.

Here is one usage example from the documentation:

>>> mode([1, 1, 2, 3, 3, 3, 3, 4])
3

Without using a TypeVar, mode could have the signature shown in Example 8-17.

from collections import Counter
from collections.abc import Iterable

def mode(data: Iterable[float]) -> float:
    pairs = Counter(data).most_common(1)
    if len(pairs) == 0:
        raise ValueError('no mode for empty data')
    return pairs[0][0]

Many uses of mode involve int or float values, but Python has other numerical types, and it is desirable that the return type follows the element type of the given Iterable. We can improve that signature using TypeVar. Let’s start with a simple, but wrong, parameterized signature:

from collections.abc import Iterable
from typing import TypeVar

T = TypeVar('T')

def mode(data: Iterable[T]) -> T:

When it first appears in the signature, the type parameter T can be any type. The second time it appears, it will mean the same type as the first.

Therefore, every iterable is consistent-with Iterable[T], including iterables of unhashable types that collections.Counter cannot handle. We need to restrict the possible types assigned to T. We’ll see two ways of doing that in the next two sections.

from collections.abc import Iterable
from decimal import Decimal
from fractions import Fraction
from typing import TypeVar

NumberT = TypeVar('NumberT', float, Decimal, Fraction)

def mode(data: Iterable[NumberT]) -> NumberT:

>>> mode(["red", "blue", "blue", "red", "green", "red", "red"])
'red'

NumberT = TypeVar('NumberT', float, Decimal, Fraction, str)

from collections.abc import Iterable, Hashable

def mode(data: Iterable[Hashable]) -> Hashable:

from collections import Counter
from collections.abc import Iterable, Hashable
from typing import TypeVar

HashableT = TypeVar('HashableT', bound=Hashable)

def mode(data: Iterable[HashableT]) -> HashableT:
    pairs = Counter(data).most_common(1)
    if len(pairs) == 0:
        raise ValueError('no mode for empty data')
    return pairs[0][0]

AnyStr = TypeVar('AnyStr', bytes, str)

The Protocol type, as presented in PEP 544—Protocols: Structural subtyping (static duck typing), is similar to interfaces in Go: a protocol type is defined by specifying one or more methods, and the type checker verifies that those methods are implemented where that protocol type is required.

In Python, a protocol definition is written as a typing.Protocol subclass. However, classes that implement a protocol don’t need to inherit, register, or declare any relationship with the class that defines the protocol. It’s up to the type checker to find the available protocol types and enforce their usage.

Here is a problem that can be solved with the help of Protocol and TypeVar. Suppose you want to create a function top(it, n) that returns the largest n elements of the iterable it:

>>> top([4, 1, 5, 2, 6, 7, 3], 3)
[7, 6, 5]
>>> l = 'mango pear apple kiwi banana'.split()
>>> top(l, 3)
['pear', 'mango', 'kiwi']
>>>
>>> l2 = [(len(s), s) for s in l]
>>> l2
[(5, 'mango'), (4, 'pear'), (5, 'apple'), (4, 'kiwi'), (6, 'banana')]
>>> top(l2, 3)
[(6, 'banana'), (5, 'mango'), (5, 'apple')]

A parameterized generic top would look like what’s shown in Example 8-19.

def top(series: Iterable[T], length: int) -> list[T]:
    ordered = sorted(series, reverse=True)
    return ordered[:length]

The problem is how to constrain T? It cannot be Any or object, because the series must work with sorted. The sorted built-in actually accepts Iterable[Any], but that’s because the optional parameter key takes a function that computes an arbitrary sort key from each element. What happens if you give sorted a list of plain objects but don’t provide a key argument? Let’s try that:

>>> l = [object() for _ in range(4)]
>>> l
[<object object at 0x10fc2fca0>, <object object at 0x10fc2fbb0>,
<object object at 0x10fc2fbc0>, <object object at 0x10fc2fbd0>]
>>> sorted(l)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '<' not supported between instances of 'object' and 'object'

The error message shows that sorted uses the < operator on the elements of the iterable. Is this all it takes? Let’s do another quick experiment:¹⁵

>>> class Spam:
...     def __init__(self, n): self.n = n
...     def __lt__(self, other): return self.n < other.n
...     def __repr__(self): return f'Spam({self.n})'
...
>>> l = [Spam(n) for n in range(5, 0, -1)]
>>> l
[Spam(5), Spam(4), Spam(3), Spam(2), Spam(1)]
>>> sorted(l)
[Spam(1), Spam(2), Spam(3), Spam(4), Spam(5)]

That confirms it: I can sort a list of Spam because Spam implements __lt__—the special method that supports the < operator.

So the T type parameter in Example 8-19 should be limited to types that implement __lt__. In Example 8-18 we needed a type parameter that implemented __hash__, so we were able to use typing.Hashable as the upper bound for the type parameter. But now there is no suitable type in typing or abc to use, so we need to create it.

Example 8-20 shows the new SupportsLessThan type, a Protocol.

from typing import Protocol, Any

class SupportsLessThan(Protocol):  
    def __lt__(self, other: Any) -> bool: ...  

A protocol is a subclass of typing.Protocol.

The body of the protocol has one or more method definitions, with ... in their bodies.

A type T is consistent-with a protocol P if T implements all the methods defined in P, with matching type signatures.

Given SupportsLessThan, we can now define this working version of top in Example 8-21.

from collections.abc import Iterable
from typing import TypeVar

from comparable import SupportsLessThan

LT = TypeVar('LT', bound=SupportsLessThan)

def top(series: Iterable[LT], length: int) -> list[LT]:
    ordered = sorted(series, reverse=True)
    return ordered[:length]

Let’s test-drive top. Example 8-22 shows part of a test suite for use with pytest. It tries calling top first with a generator expression that yields tuple[int, str], and then with a list of object. With the list of object, we expect to get a TypeError exception.

from collections.abc import Iterator
from typing import TYPE_CHECKING  

import pytest

from top import top

# several lines omitted

def test_top_tuples() -> None:
    fruit = 'mango pear apple kiwi banana'.split()
    series: Iterator[tuple[int, str]] = (  
        (len(s), s) for s in fruit)
    length = 3
    expected = [(6, 'banana'), (5, 'mango'), (5, 'apple')]
    result = top(series, length)
    if TYPE_CHECKING:  
        reveal_type(series)  
        reveal_type(expected)
        reveal_type(result)
    assert result == expected

# intentional type error
def test_top_objects_error() -> None:
    series = [object() for _ in range(4)]
    if TYPE_CHECKING:
        reveal_type(series)
    with pytest.raises(TypeError) as excinfo:
        top(series, 3)  
    assert "'<' not supported" in str(excinfo.value)

The typing.TYPE_CHECKING constant is always False at runtime, but type checkers pretend it is True when they are type checking.

Explicit type declaration for the series variable, to make the Mypy output easier to read.¹⁶

This if prevents the next three lines from executing when the test runs.

reveal_type() cannot be called at runtime, because it is not a regular function but a Mypy debugging facility—that’s why there is no import for it. Mypy will output one debugging message for each reveal_type() pseudofunction call, showing the inferred type of the argument.

This line will be flagged as an error by Mypy.

The preceding tests pass—but they would pass anyway, with or without type hints in top.py. More to the point, if I check that test file with Mypy, I see that the TypeVar is working as intended. See the mypy command output in Example 8-23.

…/comparable/ $ mypy top_test.py
top_test.py:32: note:
    Revealed type is "typing.Iterator[Tuple[builtins.int, builtins.str]]" 
top_test.py:33: note:
    Revealed type is "builtins.list[Tuple[builtins.int, builtins.str]]"
top_test.py:34: note:
    Revealed type is "builtins.list[Tuple[builtins.int, builtins.str]]" 
top_test.py:41: note:
    Revealed type is "builtins.list[builtins.object*]" 
top_test.py:43: error:
    Value of type variable "LT" of "top" cannot be "object"  
Found 1 error in 1 file (checked 1 source file)

In test_top_tuples, reveal_type(series) shows it is an Iterator[tuple[int, str]]—which I explicitly declared.

reveal_type(result) confirms that the type returned by the top call is what I wanted: given the type of series, the result is list[tuple[int, str]].

In test_top_objects_error, reveal_type(series) shows it is list[object*]. Mypy puts a * after any type that was inferred: I did not annotate the type of series in this test.

Mypy flags the error that this test intentionally triggers: the element type of the Iterable series cannot be object (it must be of type SupportsLessThan).

A key advantage of a protocol type over ABCs is that a type doesn’t need any special declaration to be consistent-with a protocol type. This allows a protocol to be created leveraging preexisting types, or types implemented in code that we do not control. I don’t need to derive or register str, tuple, float, set, etc. with SupportsLessThan to use them where a SupportsLessThan parameter is expected. They only need to implement __lt__. And the type checker will still be able do its job, because SupportsLessThan is explicitly defined as a Protocol—in contrast with the implicit protocols that are common with duck typing, which are invisible to the type checker.

The special Protocol class was introduced in PEP 544—Protocols: Structural subtyping (static duck typing). Example 8-21 demonstrates why this feature is known as static duck typing: the solution to annotate the series parameter of top was to say “The nominal type of series doesn’t matter, as long as it implements the __lt__ method.” Python’s duck typing always allowed us to say that implicitly, leaving static type checkers clueless. A type checker can’t read CPython’s source code in C, or perform console experiments to find out that sorted only requires that the elements support <.

Now we can make duck typing explicit for static type checkers. That’s why it makes sense to say that typing.Protocol gives us static duck typing.17

There’s more to see about typing.Protocol. We’ll come back to it in Part IV, where Chapter 13 contrasts structural typing, duck typing, and ABCs—another approach to formalizing protocols. In addition, “Overloaded Signatures” (Chapter 15) explains how to declare overloaded function signatures with @typing.overload, and includes an extensive example using typing.Protocol and a bounded TypeVar.

To annotate callback parameters or callable objects returned by higher-order functions, the collections.abc module provides the Callable type, available in the typing module for those not yet using Python 3.9. A Callable type is parameterized like this:

Callable[[ParamType1, ParamType2], ReturnType]

Here is an example in the context of a repl function, part of a simple interactive interpreter we’ll see in “Pattern Matching in lis.py: A Case Study”:¹⁸

def repl(input_fn: Callable[[Any], str] = input]) -> None:

During normal usage, the repl function uses Python’s input built-in to read expressions from the user. However, for automated testing or for integration with other input sources, repl accepts an optional input_fn parameter: a Callable with the same parameter and return types as input.

The built-in input has this signature on typeshed:

def input(__prompt: Any = ...) -> str: ...

The input signature is consistent-with this Callable type hint:

Callable[[Any], str]

There is no syntax to annotate optional or keyword argument types. The documentation of typing.Callable says “such function types are rarely used as callback types.” If you need a type hint to match a function with a flexible signature, replace the whole parameter list with ...—like this:

Callable[..., ReturnType]

The interaction of generic type parameters with a type hierarchy introduces a new typing concept: variance.

from collections.abc import Callable

def update(  
        probe: Callable[[], float],  
        display: Callable[[float], None]  
    ) -> None:
    temperature = probe()
    # imagine lots of control code here
    display(temperature)

def probe_ok() -> int:  
    return 42

def display_wrong(temperature: int) -> None:  
    print(hex(temperature))

update(probe_ok, display_wrong)  # type error  

def display_ok(temperature: complex) -> None:  
    print(temperature)

update(probe_ok, display_ok)  # OK  

NoReturn

This is a special type used only to annotate the return type of functions that never return. Usually, they exist to raise exceptions. There are dozens of such functions in the standard library.

def exit(__status: object = ...) -> NoReturn: ...

In Chapter 24, Example 24-6 uses NoReturn in the __flag_unknown_attrs, a method designed to produce a user-friendly and comprehensive error message, and then raise AttributeError.

The last section in this epic chapter is about positional and variadic parameters.