- Foreword
- Preface
- Programming Over Time
- Google’s Perspective
- What This Book Isn’t
- Parting Remarks
- Conventions Used in This Book
- O’Reilly Online Learning
- How to Contact Us
- Acknowledgments
- I. Thesis
- 1. What Is Software Engineering?
- Time and Change
- Hyrum’s Law
- Example: Hash Ordering
- Why Not Just Aim for “Nothing Changes”?
- Scale and Efficiency
- Policies That Don’t Scale
- Policies That Scale Well
- Example: Compiler Upgrade
- Shifting Left
- Trade-offs and Costs
- Example: Markers
- Inputs to Decision Making
- Example: Distributed Builds
- Example: Deciding Between Time and Scale
- Revisiting Decisions, Making Mistakes
- Software Engineering Versus Programming
- Conclusion
- TL;DRs
- II. Culture
- 2. How to Work Well on Teams
- Help Me Hide My Code
- The Genius Myth
- Hiding Considered Harmful
- Early Detection
- The Bus Factor
- Pace of Progress
- In Short, Don’t Hide
- It’s All About the Team
- The Three Pillars of Social Interaction
- Why Do These Pillars Matter?
- Humility, Respect, and Trust in Practice
- Blameless Post-Mortem Culture
- Being Googley
- Conclusion
- TL;DRs
- 3. Knowledge Sharing
- Challenges to Learning
- Philosophy
- Setting the Stage: Psychological Safety
- Mentorship
- Psychological Safety in Large Groups
- Growing Your Knowledge
- Ask Questions
- Understand Context
- Scaling Your Questions: Ask the Community
- Group Chats
- Mailing Lists
- YAQS: Question-and-Answer Platform
- Scaling Your Knowledge: You Always Have Something to Teach
- Office Hours
- Tech Talks and Classes
- Documentation
- Code
- Scaling Your Organization’s Knowledge
- Cultivating a Knowledge-Sharing Culture
- Establishing Canonical Sources of Information
- Staying in the Loop
- Readability: Standardized Mentorship Through Code Review
- What Is the Readability Process?
- Why Have This Process?
- Conclusion
- TL;DRs
- 4. Engineering for Equity
- Bias Is the Default
- Understanding the Need for Diversity
- Building Multicultural Capacity
- Making Diversity Actionable
- Reject Singular Approaches
- Challenge Established Processes
- Values Versus Outcomes
- Stay Curious, Push Forward
- Conclusion
- TL;DRs
- 5. How to Lead a Team
- Managers and Tech Leads (and Both)
- The Engineering Manager
- The Tech Lead
- The Tech Lead Manager
- Moving from an Individual Contributor Role to a Leadership Role
- The Only Thing to Fear Is…Well, Everything
- Servant Leadership
- The Engineering Manager
- Manager Is a Four-Letter Word
- Today’s Engineering Manager
- Antipatterns
- Antipattern: Hire Pushovers
- Antipattern: Ignore Low Performers
- Antipattern: Ignore Human Issues
- Antipattern: Be Everyone’s Friend
- Antipattern: Compromise the Hiring Bar
- Antipattern: Treat Your Team Like Children
- Positive Patterns
- Lose the Ego
- Be a Zen Master
- Be a Catalyst
- Remove Roadblocks
- Be a Teacher and a Mentor
- Set Clear Goals
- Be Honest
- Track Happiness
- The Unexpected Question
- Other Tips and Tricks
- People Are Like Plants
- Intrinsic Versus Extrinsic Motivation
- Conclusion
- TL;DRs
- 6. Leading at Scale
- Always Be Deciding
- The Parable of the Airplane
- Identify the Blinders
- Identify the Key Trade-Offs
- Decide, Then Iterate
- Always Be Leaving
- Your Mission: Build a “Self-Driving” Team
- Dividing the Problem Space
- Always Be Scaling
- The Cycle of Success
- Important Versus Urgent
- Learn to Drop Balls
- Protecting Your Energy
- Conclusion
- TL;DRs
- 7. Measuring Engineering Productivity
- Why Should We Measure Engineering Productivity?
- Triage: Is It Even Worth Measuring?
- Selecting Meaningful Metrics with Goals and Signals
- Goals
- Signals
- Metrics
- Using Data to Validate Metrics
- Taking Action and Tracking Results
- Conclusion
- TL;DRs
- III. Processes
- 8. Style Guides and Rules
- Why Have Rules?
- Creating the Rules
- Guiding Principles
- The Style Guide
- Changing the Rules
- The Process
- The Style Arbiters
- Exceptions
- Guidance
- Applying the Rules
- Error Checkers
- Code Formatters
- Conclusion
- TL;DRs
- 9. Code Review
- Code Review Flow
- How Code Review Works at Google
- Code Review Benefits
- Code Correctness
- Comprehension of Code
- Code Consistency
- Psychological and Cultural Benefits
- Knowledge Sharing
- Code Review Best Practices
- Be Polite and Professional
- Write Small Changes
- Write Good Change Descriptions
- Keep Reviewers to a Minimum
- Automate Where Possible
- Types of Code Reviews
- Greenfield Code Reviews
- Behavioral Changes, Improvements, and Optimizations
- Bug Fixes and Rollbacks
- Refactorings and Large-Scale Changes
- Conclusion
- TL;DRs
- 10. Documentation
- What Qualifies as Documentation?
- Why Is Documentation Needed?
- Documentation Is Like Code
- Know Your Audience
- Types of Audiences
- Documentation Types
- Reference Documentation
- Design Docs
- Tutorials
- Conceptual Documentation
- Landing Pages
- Documentation Reviews
- Documentation Philosophy
- WHO, WHAT, WHEN, WHERE, and WHY
- The Beginning, Middle, and End
- The Parameters of Good Documentation
- Deprecating Documents
- When Do You Need Technical Writers?
- Conclusion
- TL;DRs
- 11. Testing Overview
- Why Do We Write Tests?
- The Story of Google Web Server
- Testing at the Speed of Modern Development
- Write, Run, React
- Benefits of Testing Code
- Designing a Test Suite
- Test Size
- Test Scope
- The Beyoncé Rule
- A Note on Code Coverage
- Testing at Google Scale
- The Pitfalls of a Large Test Suite
- History of Testing at Google
- Orientation Classes
- Test Certified
- Testing on the Toilet
- Testing Culture Today
- The Limits of Automated Testing
- Conclusion
- TL;DRs
- 12. Unit Testing
- The Importance of Maintainability
- Preventing Brittle Tests
- Strive for Unchanging Tests
- Test via Public APIs
- Test State, Not Interactions
- Writing Clear Tests
- Make Your Tests Complete and Concise
- Test Behaviors, Not Methods
- Don’t Put Logic in Tests
- Write Clear Failure Messages
- Tests and Code Sharing: DAMP, Not DRY
- Shared Values
- Shared Setup
- Shared Helpers and Validation
- Defining Test Infrastructure
- Conclusion
- TL;DRs
- 13. Test Doubles
- The Impact of Test Doubles on Software Development
- Test Doubles at Google
- Basic Concepts
- An Example Test Double
- Seams
- Mocking Frameworks
- Techniques for Using Test Doubles
- Faking
- Stubbing
- Interaction Testing
- Real Implementations
- Prefer Realism Over Isolation
- How to Decide When to Use a Real Implementation
- Faking
- Why Are Fakes Important?
- When Should Fakes Be Written?
- The Fidelity of Fakes
- Fakes Should Be Tested
- What to Do If a Fake Is Not Available
- Stubbing
- The Dangers of Overusing Stubbing
- When Is Stubbing Appropriate?
- Interaction Testing
- Prefer State Testing Over Interaction Testing
- When Is Interaction Testing Appropriate?
- Best Practices for Interaction Testing
- Conclusion
- TL;DRs
- 14. Larger Testing
- What Are Larger Tests?
- Fidelity
- Common Gaps in Unit Tests
- Why Not Have Larger Tests?
- Larger Tests at Google
- Larger Tests and Time
- Larger Tests at Google Scale
- Structure of a Large Test
- The System Under Test
- Test Data
- Verification
- Types of Larger Tests
- Functional Testing of One or More Interacting Binaries
- Browser and Device Testing
- Performance, Load, and Stress testing
- Deployment Configuration Testing
- Exploratory Testing
- A/B Diff Regression Testing
- UAT
- Probers and Canary Analysis
- Disaster Recovery and Chaos Engineering
- User Evaluation
- Large Tests and the Developer Workflow
- Authoring Large Tests
- Running Large Tests
- Owning Large Tests
- Conclusion
- TL;DRs
- 15. Deprecation
- Why Deprecate?
- Why Is Deprecation So Hard?
- Deprecation During Design
- Types of Deprecation
- Advisory Deprecation
- Compulsory Deprecation
- Deprecation Warnings
- Managing the Deprecation Process
- Process Owners
- Milestones
- Deprecation Tooling
- Conclusion
- TL;DRs
- IV. Tools
- 16. Version Control and Branch Management
- What Is Version Control?
- Why Is Version Control Important?
- Centralized VCS Versus Distributed VCS
- Source of Truth
- Version Control Versus Dependency Management
- Branch Management
- Work in Progress Is Akin to a Branch
- Dev Branches
- Release Branches
- Version Control at Google
- One Version
- Scenario: Multiple Available Versions
- The “One-Version” Rule
- (Nearly) No Long-Lived Branches
- What About Release Branches?
- Monorepos
- Future of Version Control
- Conclusion
- TL;DRs
- 17. Code Search
- The Code Search UI
- How Do Googlers Use Code Search?
- Where?
- What?
- How?
- Why?
- Who and When?
- Why a Separate Web Tool?
- Scale
- Zero Setup Global Code View
- Specialization
- Integration with Other Developer Tools
- API Exposure
- Impact of Scale on Design
- Search Query Latency
- Index Latency
- Google’s Implementation
- Search Index
- Ranking
- Selected Trade-Offs
- Completeness: Repository at Head
- Completeness: All Versus Most-Relevant Results
- Completeness: Head Versus Branches Versus All History Versus Workspaces
- Expressiveness: Token Versus Substring Versus Regex
- Conclusion
- TL;DRs
- 18. Build Systems and Build Philosophy
- Purpose of a Build System
- What Happens Without a Build System?
- But All I Need Is a Compiler!
- Shell Scripts to the Rescue?
- Modern Build Systems
- It’s All About Dependencies
- Task-Based Build Systems
- Artifact-Based Build Systems
- Distributed Builds
- Time, Scale, Trade-Offs
- Dealing with Modules and Dependencies
- Using Fine-Grained Modules and the 1:1:1 Rule
- Minimizing Module Visibility
- Managing Dependencies
- Conclusion
- TL;DRs
- 19. Critique: Google’s Code Review Tool
- Code Review Tooling Principles
- Code Review Flow
- Notifications
- Stage 1: Create a Change
- Diffing
- Analysis Results
- Tight Tool Integration
- Stage 2: Request Review
- Stages 3 and 4: Understanding and Commenting on a Change
- Commenting
- Understanding the State of a Change
- Stage 5: Change Approvals (Scoring a Change)
- Stage 6: Commiting a Change
- After Commit: Tracking History
- Conclusion
- TL;DRs
- 20. Static Analysis
- Characteristics of Effective Static Analysis
- Scalability
- Usability
- Key Lessons in Making Static Analysis Work
- Focus on Developer Happiness
- Make Static Analysis a Part of the Core Developer Workflow
- Empower Users to Contribute
- Tricorder: Google’s Static Analysis Platform
- Integrated Tools
- Integrated Feedback Channels
- Suggested Fixes
- Per-Project Customization
- Presubmits
- Compiler Integration
- Analysis While Editing and Browsing Code
- Conclusion
- TL;DRs
- 21. Dependency Management
- Why Is Dependency Management So Difficult?
- Conflicting Requirements and Diamond Dependencies
- Importing Dependencies
- Compatibility Promises
- Considerations When Importing
- How Google Handles Importing Dependendencies
- Dependency Management, In Theory
- Nothing Changes (aka The Static Dependency Model)
- Semantic Versioning
- Bundled Distribution Models
- Live at Head
- The Limitations of SemVer
- SemVer Might Overconstrain
- SemVer Might Overpromise
- Motivations
- Minimum Version Selection
- So, Does SemVer Work?
- Dependency Management with Infinite Resources
- Exporting Dependencies
- Conclusion
- TL;DRs
- 22. Large-Scale Changes
- What Is a Large-Scale Change?
- Who Deals with LSCs?
- Barriers to Atomic Changes
- Technical Limitations
- Merge Conflicts
- No Haunted Graveyards
- Heterogeneity
- Testing
- Code Review
- LSC Infrastructure
- Policies and Culture
- Codebase Insight
- Change Management
- Testing
- Language Support
- The LSC Process
- Authorization
- Change Creation
- Sharding and Submitting
- Cleanup
- Conclusion
- TL;DRs
- 23. Continuous Integration
- CI Concepts
- Fast Feedback Loops
- Automation
- Continuous Testing
- CI Challenges
- Hermetic Testing
- CI at Google
- CI Case Study: Google Takeout
- But I Can’t Afford CI
- Conclusion
- TL;DRs
- 24. Continuous Delivery
- Idioms of Continuous Delivery at Google
- Velocity Is a Team Sport: How to Break Up a Deployment into Manageable Pieces
- Evaluating Changes in Isolation: Flag-Guarding Features
- Striving for Agility: Setting Up a Release Train
- No Binary Is Perfect
- Meet Your Release Deadline
- Quality and User-Focus: Ship Only What Gets Used
- Shifting Left: Making Data-Driven Decisions Earlier
- Changing Team Culture: Building Discipline into Deployment
- Conclusion
- TL;DRs
- 25. Compute as a Service
- Taming the Compute Environment
- Automation of Toil
- Containerization and Multitenancy
- Summary
- Writing Software for Managed Compute
- Architecting for Failure
- Batch Versus Serving
- Managing State
- Connecting to a Service
- One-Off Code
- CaaS Over Time and Scale
- Containers as an Abstraction
- One Service to Rule Them All
- Submitted Configuration
- Choosing a Compute Service
- Centralization Versus Customization
- Level of Abstraction: Serverless
- Public Versus Private
- Conclusion
- TL;DRs
- V. Conclusion
- Afterword
- Index