Here’s the plan: when someone uses a feature you don’t understand, simply shoot them. This is easier than learning something new, and before too long the only living coders will be writing in an easily understood, tiny subset of Python 0.9.6 <wink>.1
Tim Peters, legendary core developer and author of The Zen of Python
“Python is an easy to learn, powerful programming language.” Those are the first words of the official Python 3.10 tutorial. That is true, but there is a catch: because the language is easy to learn and put to use, many practicing Python programmers leverage only a fraction of its powerful features.
An experienced programmer may start writing useful Python code in a matter of hours. As the first productive hours become weeks and months, a lot of developers go on writing Python code with a very strong accent carried from languages learned before. Even if Python is your first language, often in academia and in introductory books it is presented while carefully avoiding language-specific features.
As a teacher introducing Python to programmers experienced in other languages, I see another problem that this book tries to address: we only miss stuff we know about. Coming from another language, anyone may guess that Python supports regular expressions, and look that up in the docs. But if you’ve never seen tuple unpacking or descriptors before, you will probably not search for them, and you may end up not using those features just because they are specific to Python.
This book is not an A-to-Z exhaustive reference of Python. Its emphasis is on the language features that are either unique to Python or not found in many other popular languages. This is also mostly a book about the core language and some of its libraries. I will rarely talk about packages that are not in the standard library, even though the Python package index now lists more than 60,000 libraries, and many of them are incredibly useful.
This book was written for practicing Python programmers who want to become proficient in Python 3. I tested the examples in Python 3.10—most of them also in Python 3.9 and 3.8. When an example requires Python 3.10, it should be clearly marked.
If you are not sure whether you know enough Python to follow along, review the topics of the official Python tutorial. Topics covered in the tutorial will not be explained here, except for some features that are new.
If you are just learning Python, this book is going to be hard to follow. Not only that, if you read it too early in your Python journey, it may give you the impression that every Python script should leverage special methods and metaprogramming tricks. Premature abstraction is as bad as premature optimization.
I recommend that everyone read Chapter 1, “The Python Data Model”. The core audience for this book should not have trouble jumping directly to any part in this book after Chapter 1, but often I assume you’ve read preceding chapters in each specific part. Think of Parts I through V as books within the book.
I tried to emphasize using what is available before discussing how to build your own.
For example, in Part I, Chapter 2 covers sequence types that are ready to use,
including some that don’t get a lot of attention, like collections.deque.
Building user-defined sequences is only addressed in Part III,
where we also see how to leverage the abstract base classes (ABCs) from collections.abc.
Creating your own ABCs is discussed even later in Part III,
because I believe it’s important to be comfortable using an ABC before writing your own.
This approach has a few advantages. First, knowing what is ready to use can save you from reinventing the wheel. We use existing collection classes more often than we implement our own, and we can give more attention to the advanced usage of available tools by deferring the discussion on how to create new ones. We are also more likely to inherit from existing ABCs than to create a new ABC from scratch. And finally, I believe it is easier to understand the abstractions after you’ve seen them in action.
The downside of this strategy is the forward references scattered throughout the chapters. I hope these will be easier to tolerate now that you know why I chose this path.
Here are the main topics in each part of the book:
Chapter 1 introduces the Python Data Model and explains why the special methods (e.g., __repr__) are the key to the consistent behavior of objects of all types. Special methods are covered in more detail throughout the book. The remaining chapters in this
part cover the use of collection types: sequences, mappings, and sets, as well as the str versus bytes split—the cause of much celebration among Python 3 users and much pain for Python 2 users migrating their codebases. Also covered are the high-level class builders in the standard library: named tuple factories and the @dataclass decorator. Pattern matching—new in Python 3.10—is covered in sections in Chapters 2, 3, and 5, which discuss sequence patterns, mapping patterns, and class patterns.
The last chapter in Part I is about the life cycle of objects: references, mutability, and garbage collection.
Here we talk about functions as first-class objects in the language: what that means, how it affects some popular design patterns, and how to implement function decorators by leveraging closures. Also covered here is the general concept of callables in Python, function attributes, introspection, parameter annotations, and the new nonlocal declaration in Python 3. Chapter 8 introduces the major new topic of type hints in function signatures.
Now the focus is on building classes “by hand”—as opposed to using the class builders covered in Chapter 5. Like any Object-Oriented (OO) language, Python has its particular set of features that may or may not be present in the language in which you and I learned class-based programming. The chapters explain how to build your own collections, abstract base classes (ABCs), and protocols, as well as how to cope with multiple inheritance, and how to implement operator overloading—when that makes sense. Chapter 15 continues the coverage of type hints.
Covered in this part are the language constructs and libraries that go beyond traditional control flow with conditionals, loops, and subroutines. We start with generators, then visit context managers and coroutines, including the challenging but powerful new yield from syntax. Chapter 18 includes a significant example using pattern matching in a simple but functional language interpreter. Chapter 19, “Concurrency Models in Python” is a new chapter presenting an overview of alternatives for concurrent and parallel processing in Python, their limitations, and how software architecture allows Python to operate at web scale. I rewrote the chapter about asynchronous programming to emphasize core language features—e.g., await, async dev, async for, and async with, and show how they are used with asyncio and other frameworks.
This part starts with a review of techniques for building classes with attributes created dynamically to handle semi-structured data, such as JSON datasets. Next, we cover the familiar properties mechanism, before diving into how object attribute access works at a lower level in Python using descriptors. The relationship among functions, methods, and descriptors is explained. Throughout Part V, the step-by-step implementation of a field validation library uncovers subtle issues that lead to the advanced tools of the final chapter: class decorators and metaclasses.
Often we’ll use the interactive Python console to explore the language and libraries. I feel it is important to emphasize the power of this learning tool, particularly for those readers who’ve had more experience with static, compiled languages that don’t provide a read-eval-print loop (REPL).
One of the standard Python testing packages, doctest, works by simulating console sessions and verifying that the expressions evaluate to the responses shown. I used doctest to check most of the code in this book, including the console listings. You don’t need to use or even know about doctest to follow along: the key feature of doctests is that they look like transcripts of interactive Python console sessions, so you can easily try out the demonstrations yourself.
Sometimes I will explain what we want to accomplish by showing a doctest before the code that makes it pass.
Firmly establishing what is to be done before thinking about how to do it helps focus our coding effort.
Writing tests first is the basis of test-driven development (TDD), and I’ve also found it helpful when teaching.
If you are unfamiliar with doctest, take a look at its documentation
and this book’s example code repository.
I also wrote unit tests for some of the larger examples using pytest—which I find easier to use and more powerful than the unittest module in the standard library.
You’ll find that you can verify the correctness of most of the code in the book by typing python3 -m doctest example_script.py or pytest in the command shell of your OS.
The pytest.ini configuration at the root of the example code repository ensures that doctests are collected and executed by the pytest command.
I have been using, teaching, and debating Python since 1998, and I enjoy studying and comparing programming languages, their design, and the theory behind them. At the end of some chapters, I have added “Soapbox” sidebars with my own perspective about Python and other languages. Feel free to skip these if you are not into such discussions. Their content is completely optional.
Covering new features—like type hints, data classes, and pattern matching—made this second edition almost 30% larger than the first. To keep the book luggable, I moved some content to fluentpython.com. You will find links to articles I published there in several chapters. Some sample chapters are also in the companion website. The full text is available online at the O’Reilly Learning subscription service. The example code repository is on GitHub.
The following typographical conventions are used in this book:
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant widthUsed for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.
Note that when a line break falls within a constant_width term, a hyphen is not added—it could be misunderstood as part of the term.
Constant width boldShows commands or other text that should be typed literally by the user.
Constant width italicShows text that should be replaced with user-supplied values or by values determined by context.
This element signifies a tip or suggestion.
This element signifies a general note.
This element indicates a warning or caution.
Every script and most code snippets that appear in the book are available in the Fluent Python code repository on GitHub at https://fpy.li/code.
If you have a technical question or a problem using the code examples, please send email to bookquestions@oreilly.com.
This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.
We appreciate, but generally do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN, e.g., “Fluent Python, 2nd ed., by Luciano Ramalho (O’Reilly). Copyright 2022 Luciano Ramalho, 978-1-492-05635-5.”
If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com.
For more than 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed.
Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit http://oreilly.com.
Please address comments and questions concerning this book to the publisher:
We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://fpy.li/p-4.
Email bookquestions@oreilly.com to comment or ask technical questions about this book.
For news and information about our books and courses, visit http://oreilly.com.
Find us on Facebook: http://facebook.com/oreilly.
Follow us on Twitter: https://twitter.com/oreillymedia.
Watch us on YouTube: http://www.youtube.com/oreillymedia.
I did not expect updating a Python book five years later to be such a major undertaking, but it was. Marta Mello, my beloved wife, was always there when I needed her. My dear friend Leonardo Rochael helped me from the earliest writing to the final technical review, including consolidating and double-checking the feedback from the other tech reviewers, readers, and editors. I honestly don’t know if I’d have made it without your support, Marta and Leo. Thank you so much!
Jürgen Gmach, Caleb Hattingh, Jess Males, Leonardo Rochael, and Miroslav Šedivý were the outstanding technical review team for the second edition. They reviewed the whole book. Bill Behrman, Bruce Eckel, Renato Oliveira, and Rodrigo Bernardo Pimentel reviewed specific chapters. Their many suggestions from different perspectives made the book much better.
Many readers sent corrections or made other contributions during the early release phase, including: Guilherme Alves, Christiano Anderson, Konstantin Baikov, K. Alex Birch, Michael Boesl, Lucas Brunialti, Sergio Cortez, Gino Crecco, Chukwuerika Dike, Juan Esteras, Federico Fissore, Will Frey, Tim Gates, Alexander Hagerman, Chen Hanxiao, Sam Hyeong, Simon Ilincev, Parag Kalra, Tim King, David Kwast, Tina Lapine, Wanpeng Li, Guto Maia, Scott Martindale, Mark Meyer, Andy McFarland, Chad McIntire, Diego Rabatone Oliveira, Francesco Piccoli, Meredith Rawls, Michael Robinson, Federico Tula Rovaletti, Tushar Sadhwani, Arthur Constantino Scardua, Randal L. Schwartz, Avichai Sefati, Guannan Shen, William Simpson, Vivek Vashist, Jerry Zhang, Paul Zuradzki—and others who did not want to be named, sent corrections after I delivered the draft, or are omitted because I failed to record their names—sorry.
During my research, I learned about typing, concurrency, pattern matching, and metaprogramming while interacting with Michael Albert, Pablo Aguilar, Kaleb Barrett, David Beazley, J. S. O. Bueno, Bruce Eckel, Martin Fowler, Ivan Levkivskyi, Alex Martelli, Peter Norvig, Sebastian Rittau, Guido van Rossum, Carol Willing, and Jelle Zijlstra.
O’Reilly editors Jeff Bleiel, Jill Leonard, and Amelia Blevins made suggestions that improved the flow of the book in many places. Jeff Bleiel and production editor Danny Elfanbaum supported me throughout this long marathon.
The insights and suggestions of every one of them made the book better and more accurate. Inevitably, there will still be bugs of my own creation in the final product. I apologize in advance.
Finally, I want to extend my heartfelt thanks to my colleagues at Thoughtworks Brazil—and especially to my sponsor, Alexey Bôas—who supported this project in many ways, all the way.
Of course, everyone who helped me understand Python and write the first edition now deserves double thanks. There would be no second edition without a successful first.
The Bauhaus chess set by Josef Hartwig is an example of excellent design: beautiful, simple, and clear. Guido van Rossum, son of an architect and brother of a master font designer, created a masterpiece of language design. I love teaching Python because it is beautiful, simple, and clear.
Alex Martelli and Anna Ravenscroft were the first people to see the outline of this book and encouraged me to submit it to O’Reilly for publication. Their books taught me idiomatic Python and are models of clarity, accuracy, and depth in technical writing. Alex’s 6,200+ Stack Overflow posts are a fountain of insights about the language and its proper use.
Martelli and Ravenscroft were also technical reviewers of this book, along with Lennart Regebro and Leonardo Rochael. Everyone in this outstanding technical review team has at least 15 years of Python experience, with many contributions to high-impact Python projects in close contact with other developers in the community. Together they sent me hundreds of corrections, suggestions, questions, and opinions, adding tremendous value to the book. Victor Stinner kindly reviewed Chapter 21, bringing his expertise as an asyncio maintainer to the technical review team. It was a great privilege and a pleasure to collaborate with them over these past several months.
Editor Meghan Blanchette was an outstanding mentor, helping me improve the organization and flow of the book, letting me know when it was boring, and keeping me from delaying even more. Brian MacDonald edited chapters in Part II while Meghan was away. I enjoyed working with them, and with everyone I’ve contacted at O’Reilly, including the Atlas development and support team (Atlas is the O’Reilly book publishing platform, which I was fortunate to use to write this book).
Mario Domenech Goulart provided numerous, detailed suggestions starting with the first early release. I also received valuable feedback from Dave Pawson, Elias Dorneles, Leonardo Alexandre Ferreira Leite, Bruce Eckel, J. S. Bueno, Rafael Gonçalves, Alex Chiaranda, Guto Maia, Lucas Vido, and Lucas Brunialti.
Over the years, a number of people urged me to become an author, but the most persuasive were Rubens Prates, Aurelio Jargas, Rudá Moura, and Rubens Altimari. Mauricio Bussab opened many doors for me, including my first real shot at writing a book. Renzo Nuccitelli supported this writing project all the way, even if that meant a slow start for our partnership at python.pro.br.
The wonderful Brazilian Python community is knowledgeable, generous, and fun. The Python Brasil group has thousands of people, and our national conferences bring together hundreds, but the most influential in my journey as a Pythonista were Leonardo Rochael, Adriano Petrich, Daniel Vainsencher, Rodrigo RBP Pimentel, Bruno Gola, Leonardo Santagada, Jean Ferri, Rodrigo Senra, J. S. Bueno, David Kwast, Luiz Irber, Osvaldo Santana, Fernando Masanori, Henrique Bastos, Gustavo Niemayer, Pedro Werneck, Gustavo Barbieri, Lalo Martins, Danilo Bellini, and Pedro Kroger.
Dorneles Tremea was a great friend (incredibly generous with his time and knowledge), an amazing hacker, and the most inspiring leader of the Brazilian Python Association. He left us too early.
My students over the years taught me a lot through their questions, insights, feedback, and creative solutions to problems. Érico Andrei and Simples Consultoria made it possible for me to focus on being a Python teacher for the first time.
Martijn Faassen was my Grok mentor and shared invaluable insights with me about Python and Neanderthals. His work and that of Paul Everitt, Chris McDonough, Tres Seaver, Jim Fulton, Shane Hathaway, Lennart Regebro, Alan Runyan, Alexander Limi, Martijn Pieters, Godefroid Chapelle, and others from the Zope, Plone, and Pyramid planets have been decisive in my career. Thanks to Zope and surfing the first web wave, I was able to start making a living with Python in 1998. José Octavio Castro Neves was my partner in the first Python-centric software house in Brazil.
I have too many gurus in the wider Python community to list them all, but besides those already mentioned, I am indebted to Steve Holden, Raymond Hettinger, A.M. Kuchling, David Beazley, Fredrik Lundh, Doug Hellmann, Nick Coghlan, Mark Pilgrim, Martijn Pieters, Bruce Eckel, Michele Simionato, Wesley Chun, Brandon Craig Rhodes, Philip Guo, Daniel Greenfeld, Audrey Roy, and Brett Slatkin for teaching me new and better ways to teach Python.
Most of these pages were written in my home office and in two labs: CoffeeLab and Garoa Hacker Clube. CoffeeLab is the caffeine-geek headquarters in Vila Madalena, São Paulo, Brazil. Garoa Hacker Clube is a hackerspace open to all: a community lab where anyone can freely try out new ideas.
The Garoa community provided inspiration, infrastructure, and slack. I think Aleph would enjoy this book.
My mother, Maria Lucia, and my father, Jairo, always supported me in every way. I wish he was here to see the book; I am glad I can share it with her.
My wife, Marta Mello, endured 15 months of a husband who was always working, but remained supportive and coached me through some critical moments in the project when I feared I might drop out of the marathon.
Thank you all, for everything.
1 Message to the comp.lang.python Usenet group, Dec. 23, 2002: “Acrimony in c.l.p”.