Between the NodeSchool San Francisco and Ann Arbor PHP MySQL groups, I’ve dedicated several years of my life to teaching others how to program. By now I’ve worked with hundreds of students, often starting with the mundane process of installing required software and configuring it. Afterwards, with a little bit of code and a whole lot of explanation, we get to the part where the student’s program runs and it all just “clicks.” I can always tell when it happens: the student smiles and they discuss the possibilities of their newly acquired skill as if it were a power-up in a video game.
My goal is to re-create that tingle of excitement for you, the reader, throughout this book. Within these pages you’ll find many hands-on examples where you get to run various backing services on your development machine and then interact with them using example Node.js application code. With that comes lots of explanation and small tangents to appease the curious.
Once you’re finished with this book, you will have installed and run many different services and, with each of these services, you will have written Node.js application code to interact with them. This book places a greater emphasis on these interactions than it does on examining Node.js application code.
JavaScript is a powerful language capable of developing both frontend and backend applications. This makes it too easy to go all-in on just learning the language while shying away from periphery technologies. The thesis of this book is that we JavaScript engineers benefit greatly by having first-hand experience with technologies that many assume only engineers using more traditional enterprise platforms like Java or .NET are familiar with.
This book won’t teach you how to use Node.js, and to benefit the most from it, you should have already written several Node.js applications and have a concrete understanding of JavaScript. That said, this book does cover some advanced and lesser-known concepts about Node.js and JavaScript, such as “The Single-Threaded Nature of JavaScript” and “The Node.js Event Loop”. You should also be familiar with the basics of HTTP, have used at least one database for persisting state, and know how easy and dangerous it is to maintain state within a running Node.js process.
Perhaps you already work at a company that has infrastructure for running backend services and you’re eager to learn how it works and how your Node.js applications can benefit from it. Or maybe you’ve got a Node.js application that you’re running as a side project and you’re tired of it crashing. You might even be the CTO of a young startup and are determined to meet the demands of your growing userbase. If any of these situations sound familiar, then this book is for you.
Node.js is often used for building frontend web applications. This book doesn’t cover any topics related to frontend development or browser concerns. A wealth of books are already available that cover such content. Instead, the goal of this book is to have you integrate backend Node.js services with various services that support modern distributed systems.
By the time you’re done reading this book, you’ll have an understanding of many technologies required to run Node.js services in a production environment. For example, what it takes to deploy and scale an application, how to make it redundant and resilient to failure, how to reliably communicate with other distributed processes, and how to observe the health of the application.
You won’t become an expert on these systems just by reading this book. The operational work required to tune and shard and deploy scalable ELK services to production, for example, isn’t touched on. However, you will understand how to run a local ELK instance, send it logs from your Node.js service, and create a dashboard for visualizing the service’s health (this is covered in “Logging with ELK”).
This book certainly doesn’t cover all of the technology used by your particular employer. Although Chapter 7 discusses Kubernetes, a technology for orchestrating the deployments of application code, your employer may instead use a different solution like Apache Mesos. Or perhaps you rely on a version of Kubernetes in a cloud environment where the underlying implementation is hidden from you. At any rate, by learning about tools in the different layers of a distributed backend service stack, you’ll more easily understand other technology stacks that you may encounter.
The following typographical conventions are used in this book:
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant widthUsed for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.
Constant width boldShows commands or other text that should be typed literally by the user.
Constant width italicShows text that should be replaced with user-supplied values or by values determined by context.
This element signifies a tip or suggestion.
This element signifies a general note.
This element indicates a warning or caution.
Supplemental material (code examples, exercises, etc.) is available for download at https://github.com/tlhunter/distributed-node.
If you have a technical question or a problem using the code examples, please send email to bookquestions@oreilly.com.
This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.
We appreciate, but generally do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Distributed Systems with Node.js by Thomas Hunter II (O’Reilly). Copyright 2020 Thomas Hunter II, 978-1-492-07729-9.”
If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com.
For more than 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed.
Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit http://oreilly.com.
Please address comments and questions concerning this book to the publisher:
We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/dist-nodejs.
Email bookquestions@oreilly.com to comment or ask technical questions about this book.
For news and information about our books and courses, visit http://oreilly.com.
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Watch us on YouTube: http://youtube.com/oreillymedia
This book was made possible thanks to the detailed technical reviews provided by the following people:
Fernando is an engineer, open source contributor, and has been leading JavaScript and Node.js communities for several years both in South America and in the US. He’s currently a Senior Software Engineer at Square, and with previous tenures at other major tech companies, such as Twilio and Groupon, he has been developing enterprise-level Node.js and scaling web applications used by millions of users for more than seven years.
Bryan is an open source JavaScript and Rust programmer and enthusiast and has worked on large enterprise systems, instrumentation, and application security. Currently he’s a Senior Open Source Software engineer at Datadog. He’s used Node.js both professionally and in personal projects since not long after its inception. He is also a Node.js core collaborator and has contributed to Node.js in many ways through several of its various Working Groups.
Julián Duque is a community leader, public speaker, JavaScript/Node.js evangelist, and an official Node.js collaborator (Emeritus). Currently working at Salesforce Heroku as a Sr. Developer Advocate and currently organizing JSConf and NodeConf Colombia, he is also helping organize JSConf México and MedellinJS, the largest JavaScript user group in Colombia with 5,000+ registered members. He is also passionate about education and has been teaching software development fundamentals, JavaScript, and Node.js through different community workshops, professional training engagements, and online platforms such as Platzi.
I’d also like to give a special thanks to those who provided me with guidance and feedback: Dan Shaw (@dshaw), Brad Vogel (@BradVogel), Matteo Collina (@matteocollina), Matt Ranney (@mranney), and Rich Trott (@trott).