Single and Double Quotes Are the Same

Around Python strings, single- and double-quote characters are interchangeable, though they must match, and must be straight quotes (beware tools that autocorrect to slanted quotes!). That is, string literals can be written enclosed in either two single or two double straight quotes—the two forms work the same and return the same type of object. For example, the following two strings, coded at the usual REPL, are identical once they are read by Python:

$ python3
>>> 'python', "python"
('python', 'python')

The reason for supporting both is that it allows you to embed a quote character of the other variety inside a string without escaping it with a backslash. You may embed a double-quote character in a string enclosed in single-quote characters, and vice versa, without having to use the escapes you’ll meet in a moment:

>>> 'python"s', "python's"                  # Mixed quotes sans escapes
('python"s', "python's")

This book generally prefers to use single quotes around strings just because they are marginally easier to read, except in cases where a single quote is embedded in the string. This is a purely subjective style choice, but Python displays strings this way, too, and most Python programmers do the same today, so you probably should too.

Note that the comma is important in the preceding code. Without it, Python automatically concatenates adjacent string literals of any kind. It is almost as simple to add a + operator between them to invoke concatenation explicitly, but adjacent literals are concatenated early when your code is read (and as you’ll learn ahead, wrapping this form in parentheses also allows it to span multiple lines for larger blocks of text that can’t use triple quotes):

>>> title = "Learning " 'Python' " 6E"      # Implicit concatenation when read
>>> title
'Learning Python 6E'

Adding commas between these strings would result in a tuple, not a string. Also notice in all of these outputs that Python prints strings in single quotes unless they embed one. If needed, you can also embed quote characters by escaping them with backslashes:

>>> 'python\'s', "python\"s"
("python's", 'python"s')

To understand why, you need to move on to learn how escapes work in general.

Escape Sequences Are Special Characters

The prior example embedded a quote inside a string by preceding it with a backslash (\). This is representative of a general pattern in strings: backslashes are used to introduce special character codings known as escape sequences.

Escape sequences let us embed characters in strings that cannot easily be inserted into a string literal or typed on a keyboard. The character \, and one or more characters following it in the string literal, are replaced with a single character in the resulting string object, which has the value specified by the escape sequence. For example, here is a five-character string that embeds a newline and a tab:

>>> s = 'a\nb\tc'

The two characters \n stand for a single character—the newline character (technically speaking, code-point value 10, which means newline in Unicode and its ASCII subset). Similarly, the sequence \t is replaced with the tab character (code point 9). The way this string looks when printed depends on how you print it. The interactive echo shows the special characters as escapes, but print interprets them instead:

>>> s
'a\nb\tc'
>>> print(s)
a
b       c

To be completely sure how many actual characters are in this string, use the built-in len function—it returns the actual number of characters in a string, regardless of how it is coded or displayed:

>>> len(s)
5

This string is five characters long: it contains an ASCII a, a newline character, an ASCII b, and so on.

Notice that the original backslash characters in the preceding examples are not really stored with the string in memory; they are used only to describe special character values to be stored in the string. For coding such special characters, Python recognizes a full set of escape code sequences, listed for reference in Table 7-2.

Some of Table 7-2’s escapes come with usage rules. Again for reference, here are the fine points:

• The \x, \u and \U escape sequences require exactly two, four, and eight hexadecimal digits, respectively. Use digits 0–9 and A–F (uppercase or lowercase) for h.

• The \o escape accepts one to three octal digits and issues a warning for values over \377 in Python 3.12 because values too big for a byte cause issues in byte strings. Use digits 0–7 for o.

• Both \u and \U are recognized only in str text-string literals (e.g., '…'), where they give a character’s Unicode code-point value. This is the code point’s decoded value.

• \x and \o escapes work in both bytes byte-string literals (b'…'), where they give a byte’s absolute value; and in str text-string literals, where they give a character’s Unicode code-point value.

• Lettered escapes like \n stand for their Unicode code points in text and their ASCII encodings in bytes, even if the two values agree (there is more on Unicode escapes in Chapter 37).

Let’s get back to running code. Some string escape sequences allow you to embed absolute values as characters of a string. When you do this, you’re really giving the code-point value of the desired character. For instance, here’s a five-character string that embeds two characters with zero values (coded as octal escapes of one digit):

>>> s = 'a\0b\0c'
>>> s
'a\x00b\x00c'
>>> len(s)
5

The character associated with code point 0 is generally known as NULL (or NUL). Importantly, in Python, a character like this does not terminate a string the way a “null byte” typically does in C. Instead, Python keeps both the string’s length and text in memory. In fact, no character terminates a string in Python. Here’s a string that is all absolute escape codes—an absolute 1 and 2 (coded in octal), followed by an absolute 3 (coded in hexadecimal), and nonprintables all:

>>> s = '\001\002\x03'
>>> s
'\x01\x02\x03'
>>> len(s)
3

Notice that Python displays nonprintable characters in hex, regardless of how they were specified. When needed, you can freely combine characters, absolute-value escapes, and the more symbolic escapes in Table 7-2. To demo, the following string contains the characters “HACK”, a tab and newline, and an absolute zero character coded in hex:

>>> S = 'H\tA\nC\x00K'
>>> S
'H\tA\nC\x00K'
>>> len(S)
7
>>> print(S)
H   A
CK

This becomes more important to know when you process binary data files in Python. Because their contents are represented as strings in your scripts, it’s OK to process binary files that contain any sorts of binary byte values. When opened in binary modes, files return raw bytes from the external file as bytes—a string variant that supports most of the syntax and tools in this chapter (you’ll find more on files and bytes in Chapters 4, 9, and 37).

Two limits in Table 7-2 merit callouts. First of all, octal escapes with values too large for a byte issue warnings as of Python 3.12 and will be errors soon—despite the fact that these escapes in text strings denote code points, not bytes:

>>> '\400'
<stdin>:1: SyntaxWarning: invalid octal escape sequence '\400'
'Ā'

This seems unlikely to break much code, but the last entry in Table 7-2 may: if Python does not recognize the character after a \ as being a valid escape code, it simply keeps the backslash in the resulting string—at least for the moment:

>>> x = 'C:\py\code'    
<stdin>:1: SyntaxWarning: invalid escape sequence '\p'

>>> x                         # Keeps \ literally (and displays it as \\)
'C:\\py\\code'
>>> len(x)                    # But not for long: don't rely on this anymore!
10

Despite this behavior being expected (and even relied upon) for three decades, it has recently been judged bad and has started issuing a warning when used. In Python 3.12, it invokes a syntax warning that doesn’t stop your program but clutters your output with nags. Worse, this will be treated as a programming-ending error in a future Python, so you should not use this going forward—and are expected to change all the code you’ve written in the past that does!

Even without this backward-incompatible Python change, though, strings that rely on this behavior seem as likely to lose backslashes in escapes as to retain them elsewhere. Instead, code literal backslashes explicitly such that they are retained in your strings in all Pythons, by either doubling them with \\ (an escape for one \), or using raw strings—the topic of the next section.

Raw Strings Suppress Escapes

As we’ve already seen, escape sequences are handy for embedding special characters in strings. Sometimes, though, the special treatment of backslashes for introducing escapes can lead to trouble. It’s surprisingly common, for instance, to see Python newcomers trying to open a file on Windows with a filename argument that looks something like this:

myfile = open('C:\new\text.dat', 'w')

thinking that they will open a file called text.dat in the directory C:\new. The problem with this is that \n is taken to stand for a newline character, and \t is replaced with a tab. In effect, the call tries to open a file named C:(newline)ew(tab)ext.dat, with usually less-than-stellar results.

This is just the sort of thing that raw strings are meant to address. If the letter r (uppercase or lowercase, but usually the latter) appears just before the opening quote of any string literal covered in this chapter, it turns off the escape mechanism. The result is that Python retains your backslashes literally, exactly as they appear in the string. Hence, to fix the filename problem, just remember to add the letter r on Windows:

myfile = open(r'C:\new\text.dat', 'w')         # Works: disable \ escapes

Alternatively, because, as noted in the preceding section, two backslashes are really an escape sequence for one backslash, you can keep your backslashes by simply doubling them up when they should be taken verbatim:

myfile = open('C:\\new\\text.dat', 'w')        # Also works: \\ means \

In fact, Python itself sometimes uses this doubling scheme when it prints strings with embedded backslashes:

>>> path = r'C:\new\text.dat'                  # Raw string: keep \s
>>> path                                       # Show as Python code
'C:\\new\\text.dat'
>>> print(path)                                # User-friendly format
C:\new\text.dat
>>> len(path)                                  # String length: \s
15

As covered in Chapter 5, the default format at the interactive prompt prints results as if they were code, and therefore escapes backslashes in the output. The print statement provides a more user-friendly format that shows that there is actually only one backslash in each spot. To verify this is the case, you can check the result of the built-in len function, which returns the number of characters in the string, independent of display formats. If you count the characters in the print(path) output, you’ll see that there really is just 1 character per backslash, for a total of 15.

Besides directory paths on Windows, raw strings are also commonly used for regular expressions in text pattern matching, supported with the re module introduced in Chapter 37. Also note that Python scripts can usually use forward slashes in directory paths on both Windows and Unix because this slash is interpreted portably (e.g., 'C:/new/text.dat' works when opening Windows files, too). Raw strings are useful for paths using native Windows backslashes, though, and any other time you want to ensure that Python will leave your \ alone. They also work for triple-quoted strings to suppress escapes (and future invalid-escape errors!) in text of the sort up next.

Triple Quotes and Multiline Strings

So far, you’ve seen single quotes, double quotes, escapes, and raw strings in action. Python also has a triple-quoted string literal format, sometimes called a block string, that is a syntactic convenience for coding multiline data. This form begins with three quotes (of either the single or double variety), is followed by any number of lines of text, and is closed with the same triple-quote sequence that opened it. Single and double quotes embedded in the string’s text may be, but do not have to be, escaped—the string does not end until Python sees three unescaped quotes of the same kind used to start the literal. For example:

>>> quip = """Python strings
...   sure have
... a lot of options"""
>>> 
>>> quip
'Python strings\n  sure have\na lot of options'

This string spans three lines. As you learned in Chapter 3, the interactive prompt changes to ... on continuation lines like this in some interfaces, but not in others. This book omits the dots in some examples to enable cut-and-paste, but don’t type them yourself if they’re listed as they are here but absent in your REPL, and extrapolate as needed.

Prompts aside, Python collects all the triple-quoted text in this example into a single multiline string, with embedded newline characters (\n) at the places where your code has physical line breaks. Notice that, as in the literal, the second line in the result has leading spaces, but the third does not—what you type is truly what you get. To see the string with the newlines interpreted, print it instead of echoing:

>>> print(quip)
Python strings
  sure have
a lot of options

In fact, triple-quoted strings will retain all the enclosed text, including any to the right of your code that you might intend as comments. So don’t do this—put your comments above or below the quoted text, or use the automatic concatenation of adjacent strings mentioned earlier, with explicit newlines if needed, and surrounding parentheses to allow line spans (you’ll learn more about this latter form when you study syntax rules in Chapters 10 and 12):

>>> menu = """
... Open           # Comments here added to string!
... Save           # Ditto
... """
>>> menu
'\nOpen           # Comments here added to string!\nSave           # Ditto\n'

>>> menu = (
... 'Open\n'       # Comments here ignored
... 'Save\n'       # But newlines not automatic
... )
>>> menu
'Open\nSave\n'

So why use triple-quoted strings? For one thing, they are useful anytime you need multiline text in your program—for example, to embed multiline error messages, or HTML, XML, JSON, or YAML code in your Python source code files. You can usually embed such blocks directly in your scripts by triple-quoting without resorting to external text files or explicit concatenation and newline characters.

Triple-quoted strings are also commonly used for docstrings (documentation strings), which are string literals that are taken as comments when they appear at specific points in your file (and are covered in full in Chapter 15). These don’t have to be triple-quoted blocks, but they usually are to allow for multiline comments and may need to be triple-quoted raw strings (e.g., r''') to avoid invalid escape errors in the future (see the 3.12 syntax warnings noted above).

Finally, triple-quoted strings are also sometimes used as a “horribly hackish” way to temporarily disable lines of code during development. Really, it’s not too horrible, and it’s actually a fairly common practice today, but it wasn’t the original intent of the literal. If you wish to turn off a few lines of code and run your script again, simply put three quotes above and below them, like this:

X = 1
"""
import os                            # Disable this code temporarily
print(os.getcwd())
"""
Y = 2

This was tagged as “hackish” because Python really might make a string out of the lines of code disabled this way, but this is probably not significant in terms of performance. For large sections of code, it’s also easier than manually adding hash marks before each line and later removing them. This is especially true if you are using a text editor that does not have support for editing Python code specifically. In Python, practicality often beats aesthetics.

Basic Operations

Let’s begin by interacting with the Python interpreter to illustrate the basic string operations listed earlier in Table 7-1. You can concatenate strings using the + operator and repeat them using the * operator:

>>> len('abc')            # Length: number of items
3
>>> 'abc' + 'def'         # Concatenation: a new string
'abcdef'
>>> 'Py!' * 4             # Repetition: like 'Py!' + 'Py!' + 'Py!' + 'Py!'
'Py!Py!Py!Py!'

The len built-in function here returns the length of a string (or any other object with a length). Formally, adding two string objects with + creates a new string object, with the contents of its operands joined, and repetition with * is like adding a string to itself a given number of times (minus one). In both cases, Python lets you create arbitrarily sized strings; there’s no need to predeclare anything in Python, including the sizes of data structures—you simply build string objects as needed and let Python manage the underlying memory space automatically, as we learned in Chapter 6.

Repetition may seem a bit obscure at first, but it comes in handy in a surprising number of contexts. For example, to print a line of 80 dashes, you can count up to 80, or let Python count for you:

>>> print('------ …more… ------')        # 80 dashes, the hard way
>>> print('-' * 80)                      # 80 dashes, the easy way

Notice that the operator overloading and polymorphism called out in Chapter 5 and earlier is at work here already: we’re using the same + and * operators that perform addition and multiplication when using numbers. Python does the correct operation because it knows the types of the objects being added and multiplied. But be careful: the rules aren’t quite as liberal as you might expect. For instance, Python doesn’t allow you to mix numbers and strings in + expressions: 'abc'+9 raises an error instead of automatically converting 9 to a string (we’ll fix this ahead).

As shown near the end of Table 7-1, you can also iterate over strings in loops using for statements, which repeat actions, and test membership for both characters and substrings with the in expression operator, which is essentially a search. For substrings, in is much like the str.find() method covered later in this chapter, but it returns a Boolean result instead of the substring’s position (don’t be alarmed if the following’s print indents your prompt; its end=' ' changes the default newline character at the end of the display to a space):

>>> myjob = 'hacker'
>>> for c in myjob:                     # Step through items, print each + ' '                
...     print(c, end=' ')               # Suppress newlines after each item
...
h a c k e r
>>> 'k' in myjob                        # Found
True
>>> 'z' in myjob                        # Not found
False
>>> 'HACK' in 'abcHACKdef'              # Substring search, no position returned
True

The for loop, previewed in Chapter 4, assigns a variable to successive items in a sequence (here, a string) and executes one or more statements (normally indented) for each item. In effect, the variable c becomes a cursor stepping across the string’s characters. Because iteration turns out to be a big idea in Python, we will discuss iteration tools like these and others listed in Table 7-1 in more detail later in this book (see Chapters 14 and 20).

Indexing and Slicing

Because strings are ordered collections (a.k.a. sequences) of characters, we can access their components by position. As introduced in Chapter 4, characters in a string are fetched by indexing—providing the numeric offset of the desired component in square brackets after the string. You get back the one-character string at the specified position.

As in most C-like languages, Python offsets start at 0 and end at one less than the length of the string (and the “start at 0” part may be a short-lived hurdle if you’re accustomed to counting from 1). Unlike C, however, Python also lets you fetch items from sequences such as strings using negative offsets. Technically, a negative offset is added to the length of a string to derive a positive offset, but you can also think of negative offsets as counting backward from the end. The following interaction demonstrates:

>>> S = 'code'
>>> S[0], S[-2]                         # Indexing from front or end
('c', 'd')
>>> S[1:3], S[1:], S[:-1]               # Slicing: extract a section
('od', 'ode', 'cod')

In this code, the first line defines a four-character string and assigns it to the name S. The next line indexes it in two ways: S[0] fetches the item at offset 0 from the left—the one-character string 'c' at the front; and S[−2] gets the item at offset 2 back from the end—or equivalently, at offset (4 + (−2)) from the front.

The last line in the foregoing example demonstrates slicing, a generalized form of indexing that returns an entire section, instead of a single item. It can be used to extract columns of data, chop off prefixes and suffixes, and more. Slicing can also be viewed as a type of parsing (decomposing content), especially when applied to strings, because it’s an easy way to extract substrings. In fact, we’ll explore slicing in the context of text parsing later in this chapter.

Slicing works like this: when you index a sequence object such as a string on a pair of offsets separated by a colon, Python returns a new object containing the contiguous section identified by the offset pair. The left offset is taken to be the lower bound (inclusive), and the right is the upper bound (noninclusive). That is, Python fetches all items from the lower bound up to but not including the upper bound and returns a new object containing the fetched items. If omitted, the left and right bounds default to 0 and the length of the object you are slicing, respectively.

For instance, in the example we just ran, S[1:3] extracts the items at offsets 1 and 2—it grabs the second and third items and stops before the fourth item at offset 3. Next, S[1:] gets all items beyond the first—the upper bound, which is not specified, defaults to the length of the string, which is off the end. Finally, S[:−1] fetches all but the last item—the lower bound defaults to 0, and −1 refers to the last item, noninclusive. In more graphic terms, indexes and slices map to cells as shown in Figure 7-1.

Figure 7-1. Indexes and slices: positives start from the left (0) and negatives from the right (-1)

All of which may seem confusing at first glance, but indexing and slicing are simple and powerful tools to use once you get the knack. Remember, if you’re unsure about the effects of a slice, try it out interactively. In the next chapter, you’ll see that it’s even possible to change an entire section of another object in one step by assigning to a slice (though not for immutables like the strings we’re studying here). For now, here’s a cheat sheet of the details for reference:

Indexing—S[I]—fetches components at offsets in sequences:

• The first item is at offset 0.

• Negative indexes mean counting backward from the end or right.

• Out-of-bounds offsets are an error.

• S[0] fetches the first item.

• S[−2] fetches the second item from the end (like S[len(S)−2]).

Slicing—S[I:J]—extracts contiguous sections of sequences:

• The upper bound is noninclusive.

• Slice bounds default to 0 and the sequence length, if omitted.

• Out-of-bounds offsets are adjusted be in bounds.

• S[1:3] fetches items at offsets 1 up to but not including 3.

• S[1:] fetches items at offset 1 through the end (the sequence length).

• S[:3] fetches items at offset 0 up to but not including 3.

• S[:−1] fetches items at offset 0 up to but not including the last item.

• S[:] fetches items at offsets 0 through the end—making a top-level copy of S.

Extended slicing—S[I:J:K]—accepts a step (or stride) K, which defaults to +1:

• Allows for skipping items and reversing order—see the next section.

The second-to-last bullet item listed here turns out to be a common technique: S[:] makes a full top-level copy of a sequence object—an object with the same value, but a distinct piece of memory (you’ll find more on copies in Chapter 9). This isn’t very useful for immutable objects like strings, but it comes in handy for objects that may be changed in place, such as lists: making a copy can avoid the side effects of shared references shown in Chapter 6.

Also per the cheat sheet, slices differ from indexes in their policy on out-of-bounds (off the end) offsets: they’re always errors in indexing, because the offset does not exist, but scaled to be in bounds in slicing, because this can be useful in programs that need to accommodate sizes flexibly:

>>> S = 'code'
>>> S[99]
IndexError: string index out of range
>>> S[1:99]
'ode'

Because we’re going to explore this oddity in an end-of-part exercise, though, we’ll cut the story short here.

>>> S = 'abcdefghijklmnop'
>>> S[1:10:2]                          # Skipping items
'bdfhj'
>>> S[::2]
'acegikmo'

>>> S = 'hello'
>>> S[::−1]                            # Reversing items
'olleh'

>>> S = 'abcedfg'
>>> S[5:1:−1]                          # Bounds roles differ
'fdec'

>>> 'code'[1:3]                       # Slicing syntax
'od'
>>> 'code'[slice(1, 3)]               # Slice objects with index syntax + object
'od'
>>> 'code'[::-1]
'edoc'
>>> 'code'[slice(None, None, -1)]
'edoc'

String Conversion Tools

One of Python’s design mottos is that it refuses the temptation to guess. As a prime example, you cannot add a number and a string together in Python, even if the string looks like a number (i.e., is all digits):

>>> '62' + 1
TypeError: can only concatenate str (not "int") to str

This is by design: because + can mean both addition and concatenation, the choice of conversion would be ambiguous—do you want '621' or 63? Instead, Python treats this as an error. In Python, magic is generally omitted if it will make your coding life more complex.

What to do, then, if your script obtains a number as a text string from a file or user interface? The trick is that you must simply employ conversion tools before you can treat a string like a number, or vice versa. For instance:

>>> int('62'), str(62)          # Convert from/to string
(62, '62')

The int function converts a string to a number, and the str function converts a number to its string representation (essentially, what it looks like when printed). Now, although you can’t mix strings and number types around operators such as +, you can manually convert operands before that operation if needed:

>>> S = '62'
>>> I = 1
>>> S + I
TypeError: can only concatenate str (not "int") to str

>>> int(S) + I            # Force addition
63

>>> S + str(I)            # Force concatenation
'621'

Similar built-in functions handle floating-point-number conversions to and from strings, if you need to mix the two in expressions:

>>> float('1.5') + 2.8
4.3
>>> '1.5' + str(2.8)
'1.52.8'

The built-in eval function introduced in Chapter 5 runs a string containing Python expression code, and so can also convert a string to any kind of object. The functions int and float convert only to numbers, but this restriction means they are usually faster (and more secure, because they do not accept arbitrary expression code). As we also saw briefly in Chapter 5, string formatting provides other ways to convert numbers to strings; more on it ahead.

>>> ord('h')               # Character => ID (code point)
104
>>> chr(104)               # ID => character (string)
'h'

>>> for c in 'hack':       # All code points in a string
...     print(c, ord(c))
... 
h 104
a 97
c 99
k 107

>>> S = '5'
>>> S = chr(ord(S) + 1)
>>> S
'6'
>>> S = chr(ord(S) + 1)
>>> S
'7'

>>> int('5')
5
>>> ord('5') - ord('0')
5

>>> 'hack' == 'hack', 'hact' > 'hack', 'hacker' > 'hack'
(True, True, True)

“Changing” Strings Part 1: Sequence Operations

Remember the term immutable sequence? As we’ve seen, being a sequence means that strings support operations like concatenation, repetition, indexing, and slicing. The immutable part means that you cannot change a string in place—for instance, by assigning to an index:

>>> S = 'text'
>>> S[0] = 'n'
TypeError: 'str' object does not support item assignment

How to modify textual information in Python, then? To change a string, you generally need to build a new string using tools such as concatenation and slicing, and assign the result back to the string’s original name if desired:

>>> S = 'text'
>>> S = S + 'ual!'           # To change a string, make a new one
>>> S
'textual!'
>>> S = S[:4] + ' processing' + S[-1]
>>> S
'text processing!'

The first example adds a substring at the end of S, by concatenation. Really, it makes a new string and assigns it back to S to save it, but you can think of this as “changing” the original string. The second example replaces three characters with many by slicing, indexing, and concatenating. As you’ll see in the next section, you can achieve similar effects with string methods like replace:

>>> S = 'text'
>>> S = S.replace('ex', 'hough')
>>> S
'thought'

Like every operation that yields a new string value, string methods generate new string objects. If you want to retain those objects, you can assign them to variable names. Whether by sequence operations or methods, generating a new string object for each string change is not as inefficient as it may sound—remember, as discussed in the preceding chapter, Python automatically garbage-collects (reclaims the space of) old unused string objects as you go, so newer objects reuse the space held by prior values. Python is usually more efficient than you might expect.

But string methods can do much more, as the next section will explain.

Method Call Syntax

As introduced in Chapter 4, methods are simply functions that are associated with and act upon particular objects. Technically, they are attributes attached to objects that happen to reference callable functions which always have an implied subject. In finer-grained detail, functions are packages of code, and method calls combine two operations at once—an attribute fetch and a call:

Attribute fetches

An expression of the form object.attribute means “fetch the value of attribute in object.”

Call expressions

An expression of the form function(arguments) means “invoke the code of function, passing zero or more comma-separated argument objects to it, and return function’s result value.”

Putting these two together allows us to call a method of an object. The method call expression:

object.method(arguments)

is evaluated from left to right—Python will first fetch the method of the object and then call it, passing in both object and the arguments. Or, in plain words, the method call expression means this:

Call method to process object with arguments.

If the method computes a result, it will also come back as the result of the entire method-call expression. As a more tangible example:

>>> S = 'hack'
>>> result = S.find('ac')     # Call the find method to look for 'ac' in string S

This mapping holds true for methods of both built-in types, as well as user-defined classes we’ll study later. As you’ll see throughout this part of the book, most objects have callable methods, and all are accessed using this same method-call syntax. To call an object method, as you’ll see in the following sections, you have to go through an existing object; methods cannot be run (and make little sense) without a subject.

All String Methods (Today)

Table 7-3 summarizes the methods and call patterns for built-in string objects in Python 3.12. These change over time, so be sure to check Python’s standard library manual for the most up-to-date list, or run a dir or help call interactively on any string or the str type name, as shown in Chapter 4.

In this table, S is a string object; optional arguments are enclosed in [] brackets; nested [] mean optional following an optional; *X and **X mean any number X; and the listed methods implement higher-level operations such as splitting and joining, case conversions, content tests, and substring searches and replacements.

As you can see, strings have many methods, and we don’t have space to cover them all here; omissions can be found in other resources when needed. To help you get started, though, let’s work through some code that demonstrates some of the most commonly used methods in action and illustrates Python text-processing basics along the way.

“Changing” Strings, Part 2: String Methods

As we’ve seen, most strings cannot be changed in place directly because they are immutable. We explored changing strings with sequence operations in the preceding section, but let’s resume that story here in the context of methods.

By way of review, to make a new text value from an existing string, you can construct a new string with sequence operations such as slicing and concatenation. For example, to replace two characters in the middle of a string, you can use code like this, much as we did in the prior section:

>>> S = 'textly!'
>>> S[:4] + 'ful' + S[-1]             # Make a new string with sequence ops
'textful!'

But, if you’re really just out to replace a substring, you can use the string replace method instead:

>>> S = 'textly!'
>>> S.replace('ly', 'ful')            # Replace all 'ly' with 'ful' in S
'textful!'

The replace method is more general than this code implies. It takes as arguments the original substring (of any length) and a new substring (of any length) to replace the original, and performs a global search and replace—subject to an optional third argument that limits the number of replacements made:

>>> '--@--@--@--'.replace('@', 'PY', 2)
'--PY--PY--@--'

In such a role, replace can be used as a tool to implement simple template replacements (e.g., in form letters). If you need to replace one fixed-size string that can occur at any offset, you can do a replacement again, or search for the substring with the string find method and then slice:

>>> S = 'xxxxPYxxxxPYxxxx'
>>> where = S.find('PY')              # Search for position
>>> where                             # Occurs at offset 4
4
>>> S = S[:where] + 'CODE' + S[(where+2):]
>>> S
'xxxxCODExxxxPYxxxx'

The find method returns the offset where a substring appears, or −1 if it is not found (it searches from the front by default, and its cousin rfind searches in reverse). As we saw earlier, this is a substring search operation just like the in expression, but find returns the position of a located substring. In this context, replace does the job easier, and can do more—in the following, replacing both multiple occurrences and multiple targets:

>>> S = 'xxxxPYxxxxPYxxxx'
>>> S.replace('PY', 'CODE', 1)        # Replace one
'xxxxCODExxxxPYxxxx'

>>> S.replace('PY', 'CODE')           # Replace all
'xxxxCODExxxxCODExxxx'

>>> 'xxxxWHATxxxxHOWxxxx'.replace('WHAT', 'CODE').replace('HOW', 'PYTHON')
'xxxxCODExxxxPYTHONxxxx'

As a reminder, replace returns a new string object each time (which is why two calls can be strung together here). Because strings are immutable, methods, like sequence operations, never really change the subject string in place—even if they are called “replace!” To save the new string object produced by a method call, assign it to a name:

>>> S = S.replace('PY', 'CODE')
>>> S
'xxxxCODExxxxCODExxxx'

The fact that concatenation operations and the replace method generate new string objects each time they are run is a potential downside of using them to change strings: each interim result must create a full-fledged object with a fresh copy of its text. If you have to apply many changes to a very large string, you might be able to improve your script’s performance by converting the string to an object that does support in-place changes:

>>> S = 'text'
>>> L = list(S)                       # Explode string into a list
>>> L
['t', 'e', 'x', 't']

The built-in list function (really, an object construction call) builds a new list out of the items in any sequence (or other iterable)—in this case, “exploding” the characters of a string into a list. Once the string is in this form, you can make multiple changes to it without generating a new copy for each change:

>>> L[0] = 'h'                        # Works for lists, not strings
>>> L[3] = '!'
>>> L
['h', 'e', 'x', '!']

After your changes, you can convert back to a string if needed (e.g., to write to a file) by using the string join method to “implode” the list back into a string:

>>> S = ''.join(L)                    # Implode back to a string
>>> S
'hex!'

The join method may look a bit backward on first encounter. Because it is a method of strings (not of lists), it is called through the desired delimiter string. join puts the strings in a list (or other iterable) together, with the delimiter between list items; in this case, it uses an empty string delimiter to convert from a list back to a string. More generally, any string delimiter and iterable of strings will do:

>>> 'PY'.join(['which', 'language', 'is', 'best', '?'])
'whichPYlanguagePYisPYbestPY?'

Though subject to Python implementation, joining substrings all at once might run faster than concatenating them individually. The mutable bytearray string noted earlier may help with efficiency too; because it can be changed in place, it offers an alternative to this list/join combo for simple kinds of byte-sized text like ASCII.

More String Methods: Parsing Text

Another common role for string methods is as a simple form of text parsing—that is, analyzing structure and extracting substrings. To extract substrings at fixed offsets, we can employ slicing techniques:

>>> line = 'aaa bbb ccc'

>>> col1 = line[:3]
>>> col2 = line[4:8]
>>> col3 = line[-3:]

>>> col1, col2, col3
('aaa', 'bbb ', 'ccc')

Here, the columns of data appear at fixed offsets and so may be sliced out of the original string. This technique passes for parsing, as long as the components of your data have known positions. If instead some sort of delimiter separates the data, you can pull out its components by splitting. This will work even if the data may show up at arbitrary positions within the string:

>>> line = 'aaa bbb     ccc'
>>> cols = line.split()
>>> cols
['aaa', 'bbb', 'ccc']

The string split method chops up a string into a list of substrings, around a delimiter string. We didn’t pass a delimiter in the prior example, so it defaults to whitespace—the string is split at groups of one or more spaces, tabs, and newlines, and we get back a list of the resulting substrings. In other applications, more tangible delimiters may separate the data. To demo, the next example splits (and hence parses) the string at commas, a separator common in some database roles (string conversion tools covered earlier can change substrings here into numbers):

>>> line = 'Python,3.12,scripting,33'
>>> line.split(',')
['Python', '3.12', 'scripting', '33']

Delimiters can be longer than a single character, too:

>>> line = 'youPYarePYaPYstringPYcoder'
>>> line.split('PY')
['you', 'are', 'a', 'string', 'coder']

Although there are limits to the parsing potential of slicing and splitting, both run fast and can handle basic text-extraction chores. Comma-separated text data is also part of the CSV file format; for a more advanced tool on this front, see also the csv module in Python’s standard library.

Other Common String Methods

Other string methods have more focused purposes—for example, to strip off whitespace at the end of a line of text, perform case conversions, test content, and test for a substring at the end or front:

>>> line = "Python's strings are awesome!\n"

>>> line.rstrip()                       # Drop whitespace (or other)
"Python's strings are awesome!"
>>> line.upper()                        # Case conversions
"PYTHON'S STRINGS ARE AWESOME!\n"
>>> line.isalpha()                      # Content tests
False
>>> line.endswith('awesome!\n')         # Suffix and prefix tests
True
>>> line.startswith('Python')
True

Alternative techniques can also sometimes be used to achieve the same results as string methods—the in membership operator can be used to test for the presence of a substring, for instance, and length and slicing operations can be used to mimic endswith:

>>> line.find('awesome') != -1          # Search via method call or expression
True
>>> 'awesome' in line
True

>>> sub = 'awesome!\n'
>>> line.endswith(sub)                  # End test via method call or slice
True
>>> line[-len(sub):] == sub
True

Note that none of the string methods accepts patterns—for pattern-based text processing, you must use the Python re standard library module, an advanced tool that will be introduced briefly in Chapter 37 but is mostly outside the scope of this text. Because of their limitations, though, string methods may run more quickly than the re module’s tools.

Again, because there are so many methods available for strings, we won’t look at every one here. You’ll see some additional string examples later in this book, but for more details you can also turn to the Python library manual and other reference resources, or simply experiment interactively on your own. As noted in Chapter 4, help(S.method) gives info for a method of any string object S; use help(str.method) if you have no S.

All that being said, one method is noticeably absent from this section’s coverage: format performs string formatting, which combines many operations in a single step. It’s also part of a larger topic in Python, which we turn to next.

String-Formatting Options

As a preview of what we’re going to explore in this section, here are today’s entries in the string-formatting race:

Formatting expression: '…%s…%s…' % (value, value)

The original technique available since Python’s inception, this form is loosely based upon the C language’s printf model and sees widespread use in much existing code. Values on the right replace targets on the left.

Formatting method: '…{}…{}…'.format(value, value)

A newer technique added in Python 3.0, this form is derived in part from a same-named tool in C#/.NET. It largely overlaps with the expression’s functionality but aims to address usage modes subjectively deemed subpar.

Formatting literal: f'…{value}…{value}…'

The very latest (so far) added in Python 3.6, this form is known as f-strings. It shares much with the method but apes a host of languages that support string interpolation—substituting inline expressions with their results.

You can also format strings manually with string methods, though it’s too cumbersome to count. And technically, an additional tool, string.Template, predates the method—and drives the formatting set’s length up to a whopping four—but it’s so scantly used that it gets less billing than the three primary options above and is relegated to a brief sidebar here (and you’d be excused for pretending it doesn’t exist at all, given the heft of the formatting toolbox!).

The following sections present all three formatting options above in turn. While it may be tempting to jump straight to whatever the blogosphere may be recommending as you read these words, these sections partly build on each other (e.g., f-strings use the method’s format specifier and assume it’s earlier coverage), so a linear read is suggested.

The String-Formatting Expression

Since string-formatting expressions are the original in this department, we’ll start with them. Python defines the % binary operator to work on strings. You may recall that this is also the remainder of division, or modulus, operator for numbers. When applied to strings, the % operator provides a simple way to format values as strings according to a format definition. It’s a much more concise way to code multiple substitutions than processing parts individually.

>>> 'There are %d ways to %s!' % (3, 'format')      # Formatting expression
'There are 3 ways to format!'

>>> option = 'expression'
>>> 'Meet the formatting %s!' % option              # String substitution
'Meet the formatting expression!'

>>> '%d %s %g you' % (1, 'formatter', 4.0)          # Type-specific substitutions
'1 formatter 4 you'
 
>>> '%s -- %s -- %s' % (42, 3.14159, [1, 2, 3])     # All types match a %s target
'42 -- 3.14159 -- [1, 2, 3]'

%[(keyname)][flags][width][.precision]typecode

>>> x = 1234
>>> res = 'integers: ...%d...%-6d...%06d' % (x, x, x)
>>> res
'integers: ...1234...1234  ...001234'

>>> x = 1.23456789
>>> x                                   # Default REPL display
1.23456789

>>> '%e | %f | %g' % (x, x, x)
'1.234568e+00 | 1.234568 | 1.23457'

>>> '%E' % x
'1.234568E+00'

>>> '%-6.2f | %05.2f | %+06.1f' % (x, x, x)
'1.23   | 01.23 | +001.2'

>>> '%s' % x, str(x)
('1.23456789', '1.23456789')

>>> '%f, %.2f, %.*f' % (1/3.0, 1/3.0, 4, 1/3.0)
'0.333333, 0.33, 0.3333'

>>> '%(qty)s more %(tool)s' % {'qty': 1, 'tool': 'formatter'}
'1 more formatter'

>>>                                           # Template with substitution targets
>>> reply = """
... Hello %(name)s!
... Welcome to %(year)s
... """
 
>>> values = {'name': 'Pat', 'year': 2024}    # Build up values to substitute
>>> print(reply % values)                     # Perform substitutions

Hello Pat!
Welcome to 2024

>>> name = 'Pat'
>>> year = 2024
>>> vars()
{'name: 'Pat', 'year': 2024, …plus built-in names set by Python…}

>>> '%(name)s from %(year)s' % vars()         # Variables are keys in vars()
'Pat from 2024'

>>> '%(value)f, %(value).2f, %(value).f' % ({'value': 1 / 3.0})
'0.333333, 0.33, 0'

The String-Formatting Method

As noted, Python 3.0 added a second way to format strings that some see as more Python specific. Unlike formatting expressions, the formatting method is not as closely based upon the C language’s “printf” model; is sometimes more explicit in intent; and avoids the value-or-tuple quirk of %: substituted values are just arguments, whether one or many.

On the other hand, the new technique still relies on “printf” concepts like type codes and formatting specifications. Moreover, it largely overlaps with formatting expressions; often yields more verbose code; and in practice can be just as complex in most roles. And if we’re all being honest, the value-or-tuple quirk of the % expression is much more of a concern in theory than practice. Luckily, the two are similar enough that many core concepts overlap.

>>> template = '{}, {}, and {}'                               # Relative position
>>> template.format('expr', 'method', 'fstring')
'expr, method, and fstring'
>>> template = '{0}, {1}, and {2}'                            # Absolute position
>>> template.format('expr', 'method', 'fstring')
'expr, method, and fstring'

>>> template = '{first}, {second}, and {third}'               # Keyword name
>>> template.format(first='expr', second='method', third='fstring')
'expr, method, and fstring'

>>> template = '{first}, {0}, and {third}'                    # Combos ({0} or {})
>>> template.format('method', first='expr', third='fstring')
'expr, method, and fstring'

>>> template = '%s, %s, and %s'                               # Equivalent %s 
>>> template % ('expr', 'method', 'fstring') 
'expr, method, and fstring'

>>> template = '%(first)s, %(second)s, and %(third)s'
>>> template % dict(first='expr', second='method', third='fstring') 
'expr, method, and fstring'

>>> '{pi}, {} and {years}'.format(62, pi=3.14, years=[1995, 2024])
'3.14, 62 and [1995, 2024]'

>>> X = '{pi}, {} and {years}'.format(62, pi=3.14, years=[1995, 2024])
>>> X
'3.14, 62 and [1995, 2024]'

>>> X.split(' and ')
['3.14, 62', '[1995, 2024]'] 

>>> Y = X.replace('and', 'but under no circumstances')
>>> Y
'3.14, 62 but under no circumstances [1995, 2024]'

>>> import sys    # Standard-library module

>>> 'This {1[kind]} runs {0.platform}'.format(sys, {'kind': 'laptop'})
'This laptop runs darwin'

>>> 'This {map[kind]} runs {sys.platform}'.format(sys=sys, map={'kind': 'phone'})
'This phone runs linux'

>>> somelist = list('HACK')
>>> somelist
['H', 'A', 'C', 'K'] 

>>> 'zero={0[0]}, two={0[2]}'.format(somelist)
'zero=H, two=C'

>>> 'first={}, last={}'.format(somelist[0], somelist[-1])    # [-1] fails in fmt
'first=H, last=K'

>>> parts = (somelist[0], somelist[-1], somelist[1:3])       # [1:3] fails in fmt
>>> 'first={}, last={}, middle={}'.format(*parts)            # Or {0}, {1}, {2}
"first=H, last=K, middle=['A', 'C']"

{[fieldname][component][!conversionflag][:formatspec]}

[[fill]align][sign][z][#][0][width][grouping][.precision][typecode]

>>> '{0:10} = {1:10}'.format('text', 123.4567)
'text       =   123.4567'

>>> '{0:>10} = {1:<10}'.format('text', 123.4567)
'      text = 123.4567  '

>>> '{1[kind]:^10} = {0.platform:^10}'.format(sys, dict(kind='laptop'))
'  laptop   =   darwin  '

>>> '{:10} = {:10}'.format('text', 123.4567)
'text       =   123.4567'

>>> '{0:e}, {1:.3e}, {2:g}'.format(3.14159, 3.14159, 3.14159)
'3.141590e+00, 3.142e+00, 3.14159'

>>> '{:f}, {:.2f}, {:06.2f}'.format(3.14159, 3.14159, 3.14159)
'3.141590, 3.14, 003.14'

>>> '{:X}, {:o}, {:b}'.format(255, 255, 255)         # Hex, octal, binary
'FF, 377, 11111111'

>>> hex(255), int('FF', 16), 0xFF                    # Other to/from hex
('0xff', 255, 255)

>>> oct(255), int('377', 8), 0o377                   # Other to/from octal
('0o377', 255, 255)
>>> bin(255), int('11111111', 2), 0b11111111         # Other to/from binary
('0b11111111', 255, 255)

>>> '{:,.2f}'.format(12345.678)
'12,345.68'

>>> '{0:,}  {0:_}  {1:_x}  {1:_b}'.format(2 ** 32, 0x1FFFF)
'4,294,967,296  4_294_967_296  1_ffff  1_1111_1111_1111_1111'

>>> '{:.4f}'.format(1 / 3.0)                         # Parameters hardcoded
'0.3333'
>>> '%.4f' % (1 / 3.0)                               # Ditto for expression
'0.3333'

>>> '{0:.{1}f}'.format(1 / 3.0, 4)                   # Take value from arguments
'0.3333'
>>> '%.*f' % (4, 1 / 3.0)                            # Ditto for expression
'0.3333'

>>> '{0:0<+12,.2f}!'.format(1234.564)
'+1,234.56000!'

>>> '{0:{1}{2}{3}{4}{5}.{6}{7}}!'.format(1234.564, 0, '<', '+', 12, ',', 2, 'f')
'+1,234.56000!'

>>> '{:{}{}{}{}{}{}{}{}}!'.format(1234.564, 0, '<', '+', 12, ',', '.', 2, 'f')
'+1,234.56000!'

>>> '{:.2f}'.format(1.2345)                          # String method
'1.23'
>>> format(1.2345, '.2f')                            # Built-in function
'1.23'
>>> '%.2f' % 1.2345                                  # Expression
'1.23'
>>> f'{1.2345:.2f}'                                  # Preview: f-string
'1.23'

The F-String Formatting Literal

If you’ve survived the formatting story this far, there’s some good news: the third and last variant is mostly just a takeoff on the second. The f-string is a text-string literal that embeds substitution values in the format string itself, rather than listing them separately. The code it uses to specify custom formatting, though, is the same as that used for the formatting method. Hence, much of what you just learned for the method applies here in full.

F-strings, added in Python 3.6, perform what’s generally called string interpolation—replacing the text of an expression with the result of running it live, when the f-string itself is run. These expressions are coded inside the f-string and can be arbitrary Python expression code. The effect isn’t functionally different from listing replacement values after a % in the expression, or as arguments in the format method. Because they embed values where they are to be substituted, though, f-strings are often shorter and may seem easier to read to some observers.

Syntactically, f-strings begin with the letter f (uppercase or lowercase, but usually the latter) before any string-literal form (single, double, or triple quotes). The f prefix may be combined with r in any order to code formatted raw strings that ignore backslashes (e.g., rf), and f-strings concatenate implicitly with an adjacent text-string literal of any kind (but use the f prefix on other concatenated literals if you want them to be f-strings too—even on continuation lines).

As a preview of Chapter 37, the f cannot be combined with byte-string prefix b (which means that the f-string, like the format method but unlike the % expression, works only for text, not bytes); and cannot be mixed with the backward-compatible and seldom-used u (which would yield prefixes inappropriate for this family-oriented text).

>>> what = 'coding'
>>> tool = 'Python'

>>> f'Learning {what} in {tool}'
'Learning coding in Python'

>>> 'Learning %s in %s' % (what, tool)           # Expression equivalent
'Learning coding in Python'

>>> 'Learning {} in {}'.format(what, tool)       # Method equivalent
'Learning coding in Python'

>>> task = f'Learning {what.upper() + '!'} in {tool + str(3.12)}'
>>> task
'Learning CODING! in Python3.12'

>>> what = 'f-strings'
>>> task                                 # F-strings built when run (only)
'Learning CODING! in Python3.12'

>>> task = f'Learning {what.upper() + '!'} in {tool + str(3.12)}'
>>> task
'Learning F-STRINGS! in Python3.12'

f'Learning {what + '!'}'         # OK as of Python 3.12

f'Learning {what + '!'}'         # An error before Python 3.12
f'Learning {what + \'!\'}'       # And this doesn't make it work

f"Learning {what + '!'}"         # OK before (and after) Python 3.12
f'''Learning {what + '!'}'''     # Ditto

f'Learning '{what + '!'}''       # An error in 3.12+
f'Learning \'{what + '!'}\''     # OK if escape quotes outside {}

>>> f'{'\n'.join([what] * 3) + '\x21'}'
'f-strings\nf-strings\nf-strings!'

>>> f'Learning {                 # Your comment here
...      what.upper() + '!'
...      } in {tool + str(3.12)}'
'Learning F-STRINGS! in Python3.12'

f'…literaltext… {expression [=] [!s, !r, or !a] [:formatspec]} …literaltext…'

>>> a = 3.14156
>>> b = 1_234_567

>>> f'{a} and {b}'                           # Defaults
'3.14156 and 1234567'

>>> f'{a:.2f} and {b:09}'                    # Decimals, padding        
'3.14 and 001234567'
 
>>> f'{a * 1000:,.2f} and {b:,} and {b:_}'   # Comma and underscore separators
'3,141.56 and 1,234,567 and 1_234_567'

>>> f'{a * 1000:e} and {b:+012,}'            # Exponents, signs, and padding
'3.141560e+03 and +001,234,567'

>>> f'{b:_X} and {b:_o} and {b // 64:_b}'    # Hex, octal, binary, underscores
'12_D687 and 455_3207 and 100_1011_0101_1010'

>>> f'{a=:e} and {b=:+012,}'                 # Labeled
'a=3.141560e+00 and b=+001,234,567'
 
>>> f'{a + 1=:e} and {b * 2=:+012,}'
'a + 1=4.141560e+00 and b * 2=+002,469,134'

>>> c = 'h\xc4ck'                            # \xc4 (a.k.a \u00c4) is non-ASCII Ä

>>> f'{c} and {c} and {c}'                   # Defaults
'hÄck and hÄck and hÄck'
 
>>> f'{c!s} and {c!r} and {c!a}'             # Display mode, two ways
"hÄck and 'hÄck' and 'h\\xc4ck'"
 
>>> f'{str(c)} and {repr(c)} and {ascii(c)}'
"hÄck and 'hÄck' and 'h\\xc4ck'"
 
>>> f'{c=!s} and {c=!r} and {c=!a}'          # Labeled
"c=hÄck and c='hÄck' and c='h\\xc4ck'"
 
>>> f'{c=!s:8} and {repr(c)} and {c:0>8}'    # Width, fill, alignment
"c=hÄck     and 'hÄck' and 0000hÄck"

>>> width = 8

>>> f'{a:.8f} and {c:0>8}'                   # Hardcoded parameters
'3.14156000 and 0000hÄck'

>>> f'{a:.{width}f} and {c:0>{width}}'       # Dynamic parameters
'3.14156000 and 0000hÄck'

>>> f'{a=:.{width}f} and {c * 2:0>{width * 3}}'
'a=3.14156000 and 0000000000000000hÄckhÄck'

>>> what = 1_234.564
>>> a, b, c, d, e, f, g, h = 0, '<', '+', 12, ',', '.', 2, 'f'
>>> 
>>> f'{what:0<+12,.2f}!'
'+1,234.56000!'
>>> 
>>> f'{what:{a}{b}{c}{d}{e}{f}{g}{h}}!'      # But don't try this at home?
'+1,234.56000!'

>>> values = dict(tool='Python', role='scripting')     # Collected values

>>> 'Use %(tool)s for %(role)s.' % values              # Expression: keys
'Use Python for scripting.'

>>> 'Use {tool} for {role}.'.format(**values)          # Method: keywords
'Use Python for scripting.'

>>> 'Use {0[tool]} for {0[role]}.'.format(values)      # Method: reused-arg keys
'Use Python for scripting.'

>>> f'Use {values['tool']} for {values['role']}.'      # F-string: expressions
'Use Python for scripting.'

>>> tool = 'Python'
>>> role = 'scripting'
>>> f'Use {tool} for {role}.'
'Use Python for scripting.'

>>> fs = """f'Use {tool} for {role}.'"""               # As if loaded from a file
>>> eval(fs)                                           # Run as code – and trust!
'Use Python for scripting.'

And the Winner Is…

The previous edition of this book went to considerable lengths (about seven pages) to show how the formatting method, newest at the time, was functionally redundant with the expression, in order to underscore the downsides of feature bloat in programming languages. Given that the number of primary formatting tools in Python has climbed from two to three since then, that message may not have entirely hit its mark. Consequently, this edition has dropped most of the rhetoric, and opted to leave this race’s call up to you.

But if you’re looking for a guideline, there is no killer argument for dismissing any formatting option out of hand for all use cases. As stated at the start of this section, f-strings may be best in most new code, on logistical grounds alone: newer is less likely to be culled by Python sooner. In fact, we’ll be using them regularly in the rest of this book where warranted, so expect more examples ahead. Even so, you will also see the expression and method often in existing code and may have to use them in roles that f-strings don’t address (e.g., for substitutions in text loaded from files).

More fundamentally, this book is not in the business of telling you what to do, and you are welcome to use any formatting option you prefer. Imposing new tools on programmers is exclusive, divisive, and probably rude. Despite norms in the software field today, the choice of development options should be yours—and yours alone—to make.

As for the bloat: change is not always bad, but it can be when it creates redundancy or incompatibility. In fact, you’ve just had a front-row seat to one of its worst consequences: N functionally equivalent options can multiply newcomers’ learning requirements by N. We’ll return to this and other perils of ego-fueled churn and convolution in software development at the end of this book. For now, we’ll close by simply noting that this stuff still matters. With any luck, a future edition won’t have yet another formatting tool to doc, and future learners won’t have yet another one to grok.

Types Share Operation Sets by Categories

As you’ve learned, strings are immutable sequences: they cannot be changed in place (the immutable part), and they are positionally ordered collections that are accessed by offset (the sequence part). It so happens that all the sequences we’ll study in this part of the book respond to the same sequence operations shown in this chapter at work on strings—concatenation, indexing, iteration, and so on. More formally, there are three major type (and hence operation) categories in Python that have this generic nature:

Numbers (integer, floating-point, decimal, fraction, others)

Support addition, multiplication, etc.

Sequences (strings, lists, tuples)

Support indexing, slicing, concatenation, etc.

Mappings (dictionaries)

Support indexing by key, etc.

Python’s byte strings mentioned at the start of this chapter fall under the general “strings” label here; sets are something of a category unto themselves (they don’t map keys to values and are not positionally ordered sequences); and we haven’t yet explored mappings on our in-depth tour (we will in the next chapter). However, many of the other types we will encounter will be similar to numbers and strings. For example, for any sequence objects X and Y:

• X + Y makes a new sequence object with the contents of both operands joined.

• X * N makes a new sequence object with N copies of the sequence operand X.

In other words, these operations work the same way on any kind of sequence, including strings, lists, tuples, and some user-defined object types. The only difference is that the new result object you get back is of the same type as the operands X and Y—if you concatenate lists, you get back a new list, not a string. Indexing, slicing, and other sequence operations work the same on all sequences, too; the type of the objects being processed tells Python which flavor of the task to perform (and if that sounds like polymorphism again, it should).

Mutable Types Can Be Changed in Place

The string’s immutable classification is an important constraint to be aware of, yet it tends to trip up new users. If an object type is immutable, you cannot change its value in place; Python raises an error if you try. Instead, you must run code to make a new object containing the new value. The major core types in Python break down as follows:

Immutables (numbers, strings, tuples, frozensets)

None of the object types in the immutable category support in-place changes, though we can always run expressions to make new objects and assign their results to variables as needed.

Mutables (lists, dictionaries, sets, bytearray)

Conversely, the mutable object types can always be changed in place with operations that do not create new objects. Although such objects can be copied manually, in-place changes support direct modification.

Generally, immutable types give some degree of integrity by guaranteeing that an object won’t be changed by another part of a program. For a refresher on why this matters, see the discussion of shared object references in Chapter 6. To see how lists, dictionaries, and tuples participate in type categories, we need to move ahead to the next chapter.