try Statement Clauses

When you write a try statement, a variety of clauses can appear after and below the try header. Table 34-1 summarizes all the possible forms as both reference and preview. We’ve already seen that except clauses catch exceptions and finally clauses run on the way out. New here, else clauses run if no exceptions are encountered, and except* clauses process exception groups and support all except forms except the empty.

Formally, a try must use at least one of the clauses in Table 34-1. There may be any number of except clauses, but you can code else only if there is at least one except, and there can be only one else and one finally. A finally can appear in the same statement as except and else, with ordering rules given later in this chapter.

We’ll explore the as var part available in some of Table 34-1’s clauses in more detail when we meet the raise statement later in this chapter because it provides access to the object raised as an exception via var. Before all that, let’s get started by examining the more common clauses of Table 34-1 more closely.

The except and else Clauses

Syntactically, the try is a compound, multipart statement. It starts with a try header line, followed by a block of (usually) indented statements, which is followed by clauses that each identify a condition to be handled and give a block of statements to handle it. In its most common form, try is coded with one or more except clauses and an optional else clause at the end. You associate the words try, except, and else by indenting them to the same level (i.e., lining them up vertically), like this:

try:
    statements              # Run this main action first
except name1:
    statements              # Run if name1 is raised during the try block
except (name2, name3):
    statements              # Run if name2 or name3 occur in the try
except name4 as var:
    statements              # Run if name4 is raised, and assign it to var
except:
    statements              # Run for all other exceptions raised
else:
    statements              # Run if no exception was raised in the try block

Semantically, the block under the try header in this statement represents the main action of the statement—the code you’re trying to run, and wrapping in exception handlers. The rest of the statement defines the handlers themselves: except clauses give handlers for exceptions raised during the try block, and the optional else clause gives a handler run if no exceptions occur in the try block.

Within a try, each except names exceptions to catch: a single exception catches just that exception, a tuple catches any exception in the tuple, and an except that omits the exception altogether matches all (or all other) exceptions. Each nonempty except can also give a variable name after as to be assigned the exception object raised by Python or raise statements; again, we’ll explore this option ahead.

try:
    …
except NameError:
    …
except IndexError:
    …
except (AttributeError, TypeError, SyntaxError):
    …

try:
    …
except (AttributeError, TypeError, SyntaxError) as What:
    …and check What…

try:
    …
except NameError:
    …                   # Handle NameError
except IndexError:
    …                   # Handle IndexError
except:
    …                   # Handle all other exceptions
else:
    …                   # Handle the no-exception case (preview)

try:
    …
except:
    …                   # Catch all possible exceptions

try:
    …
except Exception:
    …                   # Catch all possible exceptions - except exits

try:
    …run code…
except IndexError:
    …handle exception…
# Did we get here because the try failed or not?

try:
    …run code…
except IndexError:
    …handle exception…
else:
    …no exception occurred…

try:
    …run code…
    …no exception occurred…
except IndexError:
    …handle exception…

def gobad(x, y):
    return x / y

def gosouth(x):
    print(gobad(x, 0))

if __name__ == '__main__': gosouth(1)

$ python3 crashware.py
Traceback (most recent call last):
  File "/…/LP6E/Chapter34/crashware.py", line 7, in <module>
    if __name__ == '__main__': gosouth(1)
  File "/…/LP6E/Chapter34/crashware.py", line 5, in gosouth
    print(gobad(x, 0))
  File "/…/LP6E/Chapter34/crashware.py", line 2, in gobad
    return x / y
ZeroDivisionError: division by zero

$ python3
>>> import crashware
>>> crashware.gobad(1, 0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/…/LP6E/Chapter34/crashware.py", line 2, in gobad
    return x / y
           ~~^~~
ZeroDivisionError: division by zero

def kaboom(x, y):
    print(x + y)                        # Trigger TypeError

def serve(n=2):                         # Simulate long-running task
    for i in range(n):
        try:
            kaboom([1, 2], 'hack')
        except TypeError:               # Catch and recover here
            print('Hello world!')
        print('Resuming here...')       # Continue here if exception or not

if __name__ == '__main__': serve()

$ python3 kaboom.py
Hello world!
Resuming here...
Hello world!
Resuming here...

The finally Clause

The other main flavor of the try statement is for coding finalization (a.k.a. termination) actions and is really related to exceptions only incidentally. If a finally clause is included in a try, Python will always run its block of statements “on the way out” of the try statement—whether an exception occurred while the try block was running or not. That is, this clause doesn’t catch exceptions, it works around them.

When used in isolation, this flavor’s general form is this:

try:
    statements           # Run this action first
finally:
    statements           # Always run this code on the way out

When a finally appears in try, Python begins by running the statement block associated with the try header line as usual. What happens next depends on whether an exception occurs during the try block, and what other clauses are present:

Exception and match

If an exception occurs during the try block’s run and is matched by an except clause, Python first runs the matching except block and then runs the finally block. After both finish, the program then resumes below the entire try statement. The finally is also run if the except raises a new exception.

Exception and no match

If an exception occurs during the try block’s run but is not caught by an except, Python still comes back and runs the finally block, but it then propagates the exception up to a previously entered try or the top-level default handler. That is, finally is run even if an exception is raised and uncaught, but unlike an except, the finally does not terminate the exception—it continues being raised after the finally block runs.

No exception

If an exception does not occur while the try block is running, Python first runs the else block (if present) and then runs the finally block. After both finish, the program then resumes below the entire try statement. The finally is also run if the else raises a new exception.

The try/finally form is useful when you want to be completely sure that an action will happen after some code runs, regardless of the exception behavior of the code. In practice, it allows you to specify cleanup actions that always must occur, such as file closes and server disconnects where required.

Technically speaking, finally can appear in the same statement as except and else, so there is really a single try statement with many optional clauses. Because of its distinct role, though, as well as its ordering rules will meet in a moment, the finally clause may be best thought of as a distinct tool. Whether mixed or not, finally serves the same purpose—to specify cleanup actions that must always be run, regardless of any exceptions.

As you’ll also see later in this chapter, the with statement and its context managers provide an object-based way to do similar work for exit actions. Unlike finally, this statement also supports entry actions, but it is limited in scope to objects that implement the context-manager protocol it employs.

class MyError(Exception): pass

def stuff(file):
    file.write('Hello?')             # May be delayed in file buffer
    raise MyError()                  # <= Enable or disable me with a #

if __name__ == '__main__':
    file = open('temp.txt', 'w')     # Open an output file (this can fail too)
    try:
        stuff(file)                  # Raises exception
    finally:
        file.close()                 # Always close file to flush output buffers
    print('Am I reached?')           # Continue here only if no exception

$ python3 closer.py 
Traceback (most recent call last):
  File "/…/LP6E/Chapter34/closer.py", line 10, in <module>
    stuff(file)                      # Raises exception
  File "/…/LP6E/Chapter34/closer.py", line 5, in stuff
    raise MyError()                  # <= Enable or disable me with a #
MyError

$ python3 closer.py 
Am I reached?

Combined try Clauses

For the first 15 years of Python’s tenure (more or less), the try statement came in two flavors and was two separate statements—we could either use a finally to ensure that cleanup code was always run, or write except blocks to catch and recover from specific exceptions and optionally specify an else clause for when exceptions occurred.

That is, the finally clause could not be mixed with except and else. This was partly because of implementation issues, and partly because the meaning of mixing the two seemed obscure—catching and recovering from exceptions seemed a disjoint concept from performing cleanup actions.

For better or worse, the two statements eventually merged. Today, we can mix finally, except, and else clauses in the same statement—in part because of similar utility in the Java language (alas, many a programming-language mod owes to imitation). That is, the try statement in its most complete form looks like this:

try:                       # Combined try statement
    main-action
except Exception1:         # Catch specific exceptions
    handler1
except Exception2:         
    handler2
except:                    # Catch all (other) exceptions
    handler3
else:                      # No-exception handler
    handler4
finally:                   # The finally encloses all
    finally-block

The code in this statement’s main-action block is executed first, as usual. If that code raises an exception, all the except blocks are tested, one after another, for a match to the exception raised: handler1 is run for Exception1, handler2 for Exception2, and handler3 for all others. If no exception is raised, handler4 is run.

No matter what’s happened previously, the finally-block is executed once, after the main action block is exited and any handler block has been run. In fact, the code in the finally-block will be run even if an error or raise in an except or else block causes a new exception to be raised.

As outlined earlier, even in mixed usage like this, the finally clause does not end the exception—if an exception is active when the finally-block is executed, it continues to be propagated after the finally-block runs, and control jumps somewhere else in the program (to an earlier try, or to the default top-level handler). If no exception is active when the finally is run, control resumes after the entire try statement.

The net effect is that the finally is always run, regardless of whether:

• An exception occurred in the main action and was handled.

• An exception occurred in the main action and was not handled.

• No exceptions occurred in the main action.

• A new exception was triggered in one of the handlers.

Again, the finally serves to specify cleanup actions that must always occur on the way out of the try, regardless of what exceptions have been raised or handled.

try -> except -> else -> finally

try:                             # Format 1
    statements
except [type [as value]]:
    statements
[except [type [as value]]:
    statements]*
[else:
    statements]
[finally:
    statements]

try:                             # Format 2
    statements
finally:
    statements

try:                         # Nested equivalent to combined form
    try:
        main-action
    except Exception1:
        handler1
    except Exception2:
        handler2
    except:
        handler3
    else:
        handler4
finally:
    finally-block

sep = '-' * 45 + '\n'


print(sep + 'EXCEPTION RAISED AND CAUGHT')
try:
    x = 'hack'[99]
except IndexError:
    print('except run')
finally:
    print('finally run')
print('after run')


print(sep + 'EXCEPTION NOT RAISED')
try:
    x = 'hack'[3]
except IndexError:
    print('except run')
finally:
    print('finally run')
print('after run')


print(sep + 'EXCEPTION NOT RAISED, WITH ELSE')
try:
    x = 'hack'[3]
except IndexError:
    print('except run')
else:
    print('else run')
finally:
    print('finally run')
print('after run')


print(sep + 'EXCEPTION RAISED BUT NOT CAUGHT')
try:
    x = 1 / 0
except IndexError:
    print('except run')
finally:
    print('finally run')
print('after run')

$ python3 trycombos.py
---------------------------------------------
EXCEPTION RAISED AND CAUGHT
except run
finally run
after run
---------------------------------------------
EXCEPTION NOT RAISED
finally run
after run
---------------------------------------------
EXCEPTION NOT RAISED, WITH ELSE
else run
finally run
after run
---------------------------------------------
EXCEPTION RAISED BUT NOT CAUGHT
finally run
Traceback (most recent call last):
  File "/…/LP6E/Chapter34/trycombos.py", line 38, in <module>
    x = 1 / 0
ZeroDivisionError: division by zero

Raising Exceptions

To make raise more concrete, let’s turn to some examples. With built-in exceptions, the following two forms are equivalent—both raise an instance of the exception class named, but the first creates the instance implicitly:

raise IndexError             # Class (instance created)
raise IndexError()           # Instance (created in statement)

We can also create the instance ahead of time—because the raise statement accepts any kind of object reference, the following two examples raise IndexError just like the prior two:

exc = IndexError()           # Create instance ahead of time
raise exc

excs = [IndexError, TypeError]
raise excs[0]

In fact, the instance provided to raise can be had in the try that catches it too, per the next section.

The except as hook

When an exception is raised, Python sends the raised instance along with the exception. If a try includes an except with an as clause per Table 34-1, the variable it gives will be assigned the instance raised by raise or Python:

try:
    …
except IndexError as X:      # X assigned the raised instance object
    …

The as is optional in a try handler (if it’s omitted, the instance is simply not assigned to a name), but including it allows the handler to access both data in the instance and methods in the exception class.

This model works the same for user-defined exceptions we code with classes—the following, for example, passes to the exception class constructor arguments that become available in the handler through the assigned instance:

class MyExc(Exception): pass

try:
    raise MyExc('oops')      # Exception class with constructor args
except MyExc as X:           # Instance attributes available in handler
    print(X.args)            # Prints ('oops',)

Because this encroaches on the next chapter’s topic, though, we’ll defer further details until then.

Regardless of their source, exceptions are always identified by class instance objects, and at most one is active at any given time (sans the except* groups of Chapter 35). Once caught by an except clause anywhere in the program, an exception ends and won’t propagate to another try, unless it’s reraised by another raise statement or error.

Scopes and except as

We’ll study exception objects in more detail in the next chapter. Now that we’ve seen the as variable in action, though, we can finally clarify the related scope issue summarized back in Chapter 17. As mentioned in that chapter, the variable used to access an exception in the as clause of an except is localized to the except block—the variable is not available after the block exits, much like a temporary loop variable in comprehension expressions:

$ python3
>>> try:
...     1 / 0
... except Exception as X:            # The "as" localizes names to except block
...     print(X)
...
division by zero
>>> X
NameError: name 'X' is not defined

Unlike comprehension loop variables, though, this variable is removed after the except block exits. This is done because the variable would otherwise retain a reference to the runtime call stack, which would defer garbage collection and thus retain excess memory space. This removal occurs, though, even if you’re using the name for other purposes in the surrounding scope, and is a much more extreme policy than that used for comprehensions:

>>> X = 99
>>> {X for X in 'hack'}               # Comprehensions localize but don't remove
{'a', 'c', 'k', 'h'}
>>> X
99
>>> X = 99
>>> try:
...     1 / 0
... except Exception as X:            # But "as" localizes _and_ removes on exit!
...     print(X)
...
division by zero
>>> X                                 # Where did my X go? – an odd boundary case
NameError: name 'X' is not defined

Because of this, you should generally use unique variable names in your try statement’s except clauses, even if they are localized by scope. If you do need to reference the exception instance after the try statement, simply assign it to another name that won’t be automatically removed:

>>> try:
...     1 / 0
... except Exception as X:            # Python removes this reference
...     print(X)
...     saveit = X                    # Assign exc to retain exc if needed
...
division by zero
>>> X
NameError: name 'X' is not defined
>>> saveit
ZeroDivisionError('division by zero',)

Propagating Exceptions with raise

The raise statement is a bit more feature-rich than we’ve seen thus far. For example, a raise that does not list an exception to raise simply reraises the currently active exception. This form is typically used if you need to catch and handle an exception but don’t want the exception to die in your handler:

>>> try:
...     raise IndexError('code')         # Exceptions remember arguments
... except IndexError:
...     print('propagating')
...     raise                            # Reraise most recent exception
...
propagating
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
IndexError: code

Running a raise this way reraises the exception and propagates it to a higher handler (or the default handler at the top, which stops the program with a standard error message). Notice how the argument we passed to the exception class shows up in the error messages; you’ll learn why this happens in the next chapter.

Exception Chaining: raise from

Exceptions can sometimes be triggered in response to other exceptions—both deliberately and by new program errors. To support full disclosure in such cases, Python also allows raise statements to have an optional from clause:

raise newexception from otherexception

When the from is used in an explicit raise request, the expression following from specifies another exception class or instance to attach to the __cause__ attribute of the new exception being raised. If the raised exception is not caught, Python prints both exceptions as part of the standard error message:

>>> try:
...     1 / 0
... except Exception as E:
...     raise TypeError('Bad') from E             # Explicitly chained exceptions
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ZeroDivisionError: division by zero

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
TypeError: Bad

When an exception is raised implicitly by a program error inside an exception handler, a similar procedure is followed automatically: the previous exception is attached to the new exception’s __context__ attribute and is again displayed in the standard error message if the exception goes uncaught:

>>> try:
...     1 / 0
... except:
...     badname                                   # Implicitly chained exceptions
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ZeroDivisionError: division by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
NameError: name 'badname' is not defined

In both cases, because the original exception objects thus attached to new exception objects may themselves have attached causes, the causality chain can be arbitrarily long, and is displayed in full in error messages. That is, error messages might give more than two exceptions. The net effect in both explicit and implicit chaining contexts is to allow programmers to know all exceptions involved when one exception triggers another:

>>> try:
...     try:
...         raise IndexError()
...     except Exception as E:
...         raise TypeError() from E
... except Exception as E:
...     raise SyntaxError() from E
...
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
IndexError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 5, in <module>
TypeError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 7, in <module>
SyntaxError: None

Code like the following similarly displays three exceptions, though implicitly triggered by handler errors (its separator lines are “During handling of the above exception…” instead of “The above exception was the direct cause…”):

try:
    try:
        1 / 0
    except:
        badname
except:
    open('nonesuch')

Exception chains impact error displays, but do not affect the way that exceptions are named and caught in try statements: chains are simply recorded in exception-object attributes which may be inspected as usual where useful.

Like the combined try, chained exceptions are similar to utility in other languages (including Java and C#) though it’s not clear which languages were borrowers. In Python, it’s not unusual to see exception chains in error messages but it is uncommon to create them explicitly with raise, so we’ll defer to Python’s manuals for more details.

As a footnote note on this topic, though, Python also provides a way to stop exceptions from chaining: a raise from None allows the display of the chained exception context to be disabled when needed. This makes for less cluttered error messages in applications that convert between exception types while processing exception chains.

Example: Trapping Constraints (but Not Errors!)

Here’s a less politically charged example of assert in action. Assertions are typically used to verify program conditions during development. When displayed, their error message text automatically includes source code line information and the value listed in the assert statement. Consider the file asserter.py in Example 34-5.

def f(x):
    assert x < 0, 'x must be negative'
    return x ** 2

Running this normally triggers the assertion error for positive numbers, but running with -O does not:

$ python3
>>> import asserter
>>> asserter.f(-3)
9
>>> asserter.f(3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/…/LP6E/Chapter34/asserter.py", line 2, in f
    assert x < 0, 'x must be negative'
AssertionError: x must be negative

$ python3 -O
>>> import asserter
>>> asserter.f(3)
9

It’s important to keep in mind that assert is mostly intended for trapping user-defined constraints, not for catching genuine programming errors. Because Python traps programming errors itself, there is usually no need to code assert to catch things like out-of-bounds indexes, type mismatches, and zero divides:

def reciprocal(x):
    assert x != 0              # A generally useless assert!
    return 1 / x               # Python checks for zero automatically

Such assert use cases are usually superfluous—because Python raises exceptions on errors automatically, you might as well let it do the job for you. As a rule, you normally don’t need to do error checking explicitly in your own code.

Of course, there are exceptions to most rules. As suggested earlier in the book, if a function has to perform long-running or unrecoverable actions before it reaches the place where an exception will be triggered, you still might want to test for errors. Even in this case, though, be careful not to make your tests overly specific or restrictive, or you will limit your code’s utility.

For another example of common assert usage, see the abstract superclass example in Chapter 29; there, we used assert to make calls to undefined methods fail with a message. It’s a rare but useful tool.

Basic with Usage

The basic format of the with statement looks like this, with an optional part in square brackets here:

with expression [as variable]:
    with-block

The statements of the nested with-block are the main action to be run here. The expression is assumed to return an object that supports the context-management protocol (more on this protocol in a moment). This object may also return a value that will be assigned to the name variable if the optional as clause is present.

Note that the variable is not necessarily assigned the result of the expression; the result of the expression is the object that supports the context protocol, and the variable may be assigned something else intended to be used inside the with-block. The object returned by the expression may then run startup code before the block is started, as well as termination code after the block is done—whether the block raised an exception or not.

As noted, some built-in Python objects have been augmented to support the context-management protocol, and so can be used with the with statement. For example, file objects (covered in Chapter 9) have a context manager that automatically closes the file after the with block regardless of whether an exception is raised, and regardless of if or when the version of Python running the code may close automatically. In abstract code:

with open('somefile.txt') as myfile:
    for line in myfile:
        print(line)

Here, the call to open returns a simple file object that is assigned to the name myfile. We can use myfile with the usual file tools—in this case, the file iterator reads line by line in the for loop.

However, the open result also supports the context-management protocol used by the with statement. After this with statement has run, the context management machinery guarantees that the file object referenced by myfile is automatically closed, even if the for loop raised an exception while processing the file.

Although file objects may be automatically closed on garbage collection, it’s not always straightforward to know when that will occur, especially when using alternative Python implementations. The with statement in this role is an alternative that allows us to be sure that the close will occur automatically after execution of a specific block of code.

As covered earlier, we can achieve a similar effect with the more general and explicit try/finally idiom, but it requires three more lines of administrative code in this case (four instead of just one):

myfile = open('somefile.txt')
try:
    for line in myfile:
        print(line)
finally:
    myfile.close()

Of course, we could skip both statements, but our file may not be closed if an exception is raised during the for loop, and this can matter in long-running programs (we’ll revisit such tradeoffs in Chapter 36).

As another example, we won’t cover Python’s multithreading modules in this book, but the lock and condition synchronization objects they define may also be used with the with statement because they support the context-management protocol—in this case adding both entry and exit actions around a block. After importing threading:

lock = threading.Lock()   
with lock:
    …access shared resources…

Here, the context management machinery guarantees that the lock is automatically acquired before the block is executed and released once the block is complete, regardless of exception outcomes.

Finally, the decimal module introduced in Chapter 5 also uses context managers to simplify saving and restoring the current decimal context, which specifies the precision and rounding characteristics for calculations:

with decimal.localcontext() as ctx:
    ctx.prec = 2
    x = decimal.Decimal('1.00') / decimal.Decimal('3.00')

After this statement runs, the current thread’s context manager state is automatically restored to what it was before the statement began. To do the same with a try/finally, we would need to save the context before and restore it manually after the nested block.

The Context-Management Protocol

Although some built-in types come with context managers, we can also write new ones of our own. To implement context managers, classes use special methods that fall into the operator-overloading category to tap into the with statement. The interface expected of objects used in with statements is somewhat complex, and most programmers only need to know how to use existing context managers. For tool builders who might want to write new application-specific context managers, though, let’s take a quick look at what’s involved.

Here’s how the with statement actually works:

1. The expression is evaluated, resulting in an object known as a context manager that must have __enter__ and __exit__ methods.

2. The context manager’s __enter__ method is called. The value it returns is assigned to the variable in the as clause if present, or simply discarded otherwise.

3. The code in the nested with block is executed.

4. If the with block raises an exception, the context manager’s __exit__(type, value, traceback) method is called with the exception details. These are the same three values returned by sys.exc_info, described in the Python manuals and later in this part of the book. If this method returns a false value, the exception is reraised; otherwise, the exception is terminated. The exception should normally be reraised so that it is propagated outside the with statement after __exit__ returns.

5. If the with block does not raise an exception, the __exit__ method is still called, but its type, value, and traceback arguments are all passed in as None, and its return value is ignored.

Let’s look at a quick demo of the protocol in action. The file withas.py in Example 34-6 defines a context-manager object that simply traces the entry and exit of the with block in any with statement it is used for.

"A context manager that traces entry and exit of any with statement's block"

class TraceBlock:
    def message(self, arg):
        print('running ' + arg)

    def __enter__(self):
        print('[starting with block]')
        return self

    def __exit__(self, exc_type, exc_value, exc_tb):
        if exc_type is None:
            print('[exited normally]\n')
        else:
            print(f'[propagating exception: {exc_type}]')
            return False

if __name__ == '__main__':
    with TraceBlock() as action:
        action.message('test 1')
        print('reached')

    with TraceBlock() as action:
        action.message('test 2')
        raise TypeError
        print('not reached')

Notice that this class’s __exit__ method returns False to propagate the exception; deleting the return statement would have the same effect, as the default None return value of functions is false by definition, but explicit is generally better in coding. Also notice that the __enter__ method returns self as the object to assign to the as variable; in other use cases, this might return a completely different object instead.

When run, this module’s self-test code uses its context manager to trace the entry and exit of two with statement blocks. The net effect automatically invokes the manager’s __enter__ and __exit__ methods:

$ python3 withas.py
[starting with block]
running test 1
reached
[exited normally]

[starting with block]
running test 2
[propagating exception: <class 'TypeError'>]
Traceback (most recent call last):
  File "/…/LP6E/Chapter34/withas.py", line 25, in <module>
    raise TypeError
TypeError

Context managers can also utilize OOP state information and inheritance, but are somewhat advanced devices meant for tool builders, so we’ll skip additional details here. See Python’s standard manuals for the full story—including its coverage of the contextlib standard module that provides additional tools for coding context managers.

Also remember that the try/finally combination provides support for termination-time activities too, and is generally sufficient in roles that don’t warrant coding classes to support the with statement’s protocol.

Multiple Context Managers

There’s with statement has one last card to turn over: it may also specify multiple (sometimes called “nested”) context managers with comma syntax. For example, in the following code snippet that selects lines by substring, both files’ exit actions are automatically run to close the files when the statement block exits, regardless of exception outcomes:

with open('lines.txt') as input, open('matches.txt', 'w') as output:
    for line in input:
        if 'somekey' in line:
            output.write(line)

Any number of context manager items may be listed, and multiple items work the same as nested with statements. That is, the following hypothetical code:

with A() as a, B() as b:
    statements

is equivalent to (and possibly simpler than) the following:

with A() as a:
    with B() as b:
        statements

The net effect is that each context manager’s entry and exit method is run in turn on block entry and exit, and exceptions in the block are caught automatically and possibly reraised on with exit at the discretion of the outermost manager’s exit method. Multiple file context managers, for instance, will all be run to open files on entry, and close them on exit—exception or not.

The Termination-Handlers Shoot-Out

You can find more info on context managers in Python’s docs. Rather than getting more detailed here, let’s close out this chapter with a quick look at this extension in action and a vetting of its roles. Using newer and redundant tools like with doesn’t in and of itself prove intelligence, and it’s important to understand the tradeoffs such options imply.

First up, the following codes a parallel lines scan of files located in this book’s examples package. It uses with to open two files at once and then reads and zips together their next-line pairs on each iteration of a for loop. Thanks to the file object’s context manager, there’s no need to manually catch exceptions or close files when finished:

>>> with open('lines1.txt') as file1, open('lines2.txt') as file2:
        for pair in zip(file1, file2):
            print(pair)

('hack\n', 'HACK\n')
('code\n', 'GOOD\n')
('well\n', 'CODE\n')

You might also use this coding structure to do a lines comparison of two text files. The following simply replaces the former’s print with an if for a comparison operation, and adds an enumerate for automatic line numbers:

>>> with open('lines1.txt') as file1, open('lines2.txt') as file2:
        for (linenum, (line1, line2)) in enumerate(zip(file1, file2)):
            if line1.lower() != line2.lower():
                print(f'{linenum} => {line1!r} != {line2!r}')
 
1 => 'code\n' != 'GOOD\n'
2 => 'well\n' != 'CODE\n'

That said, with isn’t all that useful in the preceding examples when using CPython, because input file objects don’t require a buffer flush, and file objects are closed automatically when garbage collected if still open. Moreover, exceptions in the with block are still propagated outside the statement if not explicitly caught. Hence, the temporary files would be auto-closed immediately and exception behavior would be the same for simpler code like this:

for pair in zip(open('lines1.txt'), open('lines2.txt')):     # Same if auto close
    print(pair)                                              # Ditto for != test

On the other hand, some of the alternative Pythons of Chapter 2 may use different garbage collectors that require direct closes, to avoid taxing system resources. In addition, output files may require closes to ensure that any buffered content is transferred to disk so it’s available for opens in later code.

The following lines filter code addresses both concerns, by automatically closing files on statement exit, exception or not (it also uses parentheses and line splits after with available as of Python 3.10, and omits write counts for brevity):

>>> with (open('lines1.txt') as input, 
          open('uppers.txt', 'w') as output):
        for line in input:
            output.write(line.upper())
   
>>> print(open('uppers.txt').read())          # File content is available here
HACK
CODE
WELL

Still, in simple scripts, we can often just open files in separate statements and close after processing if needed. There’s no point in catching an exception if it means your program is out of business anyhow, and closes are required to flush output buffers only if files will be reopened by later code—which may never be reached after exceptions anyhow:

input  = open('lines1.txt')
output = open('uppers.txt', 'w')
for line in input:                            # Same effect if files auto close,
    output.write(line.upper())                # and file is not reopened ahead

Nevertheless, in programs that must both continue after exceptions and close output files for later use in the same program (or REPL) run, the with avoids an equivalent try/finally combination that may be more obvious to some readers, but also requires noticeably more code—eight lines instead of four, quantitatively speaking:

input  = open('lines1.txt')
output = open('uppers.txt', 'w')
try:                                         # Same but explicit close on errors
    for line in input:
        output.write(line.upper())
finally:
    input.close()                            # Ensure output file is complete
    output.close()                           # Whether exception occurs or not

Even so, the try/finally is a single tool that applies to all finalization cases and makes code explicit. The with can be more concise for context-manager users, but applies only to objects that implement its complex protocol, relies on implicit “magic” that obscures meaning, and adds redundancy that doubles the required knowledge base of programmers. As usual, you’ll have to weigh these tools’ tradeoffs for yourself.