Chapter 6. Infrastructure Components

A key part of system design is grouping elements and defining relationships among those groups. We define these groupings as architectural units, or components, that we can use in diagrams and discussions. Components are how we implement a design. Some common components for software design include applications, microservices, libraries, and classes, which span various levels of a system. This chapter describes components that we can use to discuss system designs with Infrastructure as Code.

Our industry does not widely agree on the definitions of the components relevant for Infrastructure as Code, or what to call them. So for the purposes of this book, I’ve defined four types of components: IaaS resources, code libraries, deployable infrastructure stacks (“stacks” for short), and infrastructure compositions. Each of these covers a different level of scope, usually by aggregating lower-level structures. Code libraries and stacks aggregate multiple IaaS resources. A stack may optionally aggregate code libraries. An infrastructure composition aggregates multiple stacks.

Your Terminology May Vary

Your infrastructure probably uses different terminology for its components, so I’ll list some common tools and the terms they use for these component types in the following text. I use these terms throughout this book as a common language that isn’t specific to any particular tool. You should be able to adapt and apply these patterns and approaches in this book to your own tools and system.

Both code libraries and stacks are defined and implemented by the infrastructure tool you use, even if the tool uses different names for them. For example, Terraform calls code libraries modules, CDK calls them Level 3 constructs, and Pulumi calls them component resources. Although Terraform doesn’t use a term to describe deployment-level components, it correlates with projects (for source code), and its state files correlate with a provisioned stack. Pulumi and CDK both use the term stack, while Crossplane has compositions. The concept of infrastructure compositions is an emerging one, so this component is least likely to be familiar. A Pulumi project can define multiple stacks, and a Terraform stack can contain multiple projects.

The Infrastructure Components

The four components used most in this book are summarized in Table 6-1. I’ve listed some of the terms that vendors use for these concepts to help you to understand how these may be relevant to the particular toolchain you use.

Table 6-1. Infrastructure components by scope
Design scope Component Description Examples

High-level

Infrastructure composition

A collection of deployable infrastructure stacks. Defines dependencies and integrations among stacks and the orchestration of their deployment and lifecycle. Often designed and presented based on the capabilities they provide to workloads.

Pulumi project
Terraform stack
Terragrunt service
Atmos stack

Mid-level

Infrastructure deployment stack

A complete collection of infrastructure resources deployed as a unit. Designed around deployment concerns.

CloudFormation stack
CDK stack
Azure deployment stack
Pulumi stack (deployment context)
Terraform or OpenTofu project (in the code context)
workspace (in the deployment context)
Terraform stack component as part of a composition (which is, confusingly, called a stack)
Crossplane composition
Gruntwork unit

Low-level

Code library

Infrastructure resources grouped by how their code is shared and reused across stacks.

Bicep module
CDK Level 3 construct
CloudFormation module
Pulumi component resource
Terraform module
OpenTofu module

Primitive

IaaS resource

The smallest unit of infrastructure that can be independently defined and provisioned.

CDK Level 1/2 construct, Terraform, Pulumi,
and ACK resource

You may have picked up hints in the descriptions of how each of these components’ relevance varies with the infrastructure code lifecycle described in Chapter 4. In the code editing stage, we write code that addresses IaaS resources. IaaS resources are abstracted into higher-level components when the code is built and distributed, and then manifested in the IaaS platform during deployment. Resources are the only components meaningfully visible at runtime, where their membership in modules, stacks, and compositions are visible only as tags or labels, if at all. An exception is stacks managed by the IaaS platform, as with AWS CloudFormation.

Before using these components in an infrastructure design, it’s important to understand the purpose the infrastructure will serve, which is to run workloads. Understanding how to think about workloads sets the scene for how to think about infrastructure components.

The Start of Infrastructure Design: Workloads

The starting point for designing infrastructure is understanding the workloads that it will support. A good way to begin an infrastructure design project is by engaging with the people responsible for the software. Collaborate with them to create a picture of the workloads and the infrastructure resources needed to support them.

A workload is software that runs on infrastructure. A workload could be a user-facing application, a backend business service, a microservice, or even a platform service such as a monitoring server.

Figure 6-1 shows some workloads for the FoodSpin restaurant service.

iac3 0601
Figure 6-1. Workloads for FoodSpin

FoodSpin has a website frontend and a mobile application. These share services for product browsing, searching, shopping carts, and checkout, among others. The mobile applications communicate with a hosted backend for frontend (BFF) service that connects with the shared services. Figure 6-2 adds some of the application runtime infrastructure services needed to support a subset of these workloads.

iac3 0602
Figure 6-2. Infrastructure resources to support the workloads

The diagram shows infrastructure capabilities needed specifically for each software service, such as static website content storage for the website, and separate database instances for the menu service and the ordering service. The diagram also shows shared networking and container cluster capabilities that are used by all the workloads.

An infrastructure capability diagram like this one shows the infrastructure domain at a high level, without getting into the specific implementation details. This gives us the first step in designing the infrastructure to run the software for FoodSpin’s business. The next step is to design the higher-level architecture components to implement these infrastructure capabilities.

Infrastructure Compositions

An infrastructure composition is a collection of IaaS resources organized around a workload-relevant concern. Infrastructure compositions are typically used to define the integration of multiple infrastructure stacks. The compositions may define configuration values for the stacks, and integration points between stacks that have dependencies on one another.

The contents of an infrastructure composition and the way it is presented for configuration and use should make sense to its users, who are usually the teams responsible for configuring, deploying, and managing the applications and services that use the infrastructure. In contrast, infrastructure stacks are grouped around technical concerns, especially how IaaS resources should be grouped for provisioning. Figure 6-3 shows the contrasting concerns of compositions and stacks.

iac3 0603
Figure 6-3. The contrasting concerns for infrastructure compositions and stacks

The diagram uses the menu service as an example workload and the Menu Service Hosting composition defines the infrastructure specific to that service. The composition includes three infrastructure stacks.

The infrastructure composition is the least mature type of these infrastructure components, in terms of support by infrastructure tools. In the absence of vendor-supported composition features, many infrastructure engineering teams build their own scripts and tools to deploy and integrate infrastructure stacks, as described in “Using Delivery Orchestration Scripts”. Other teams effectively use stacks as infrastructure compositions, deploying all their infrastructure together.

Chapter 7 describes patterns and antipatterns for organizing resources across one or more stacks.

Infrastructure Deployment Stacks

An infrastructure deployment stack, or just stack, is a collection of IaaS resources defined, created, and modified as an independent, complete unit. In the terminology defined by the authors of Building Evolutionary Architectures, an infrastructure deployment stack is an architectural quantum, “an independently deployable component with high functional cohesion.” As with software deployment architecture, decisions around how large stacks should be and how to group resources within them have a big impact on how easy it is to deliver, update, and manage a system.

Figure 6-4 shows the progress of an infrastructure stack in the infrastructure code processing model described in “Infrastructure Code Processing”.

iac3 0604
Figure 6-4. Stacks in the infrastructure code processing model

The stack project contains the code that specifies the resources in the stack. In the Assemble substep, the stack tool (probably one of the stack-oriented infrastructure tools listed in “Infrastructure as Code Tools” such as Terraform, Pulumi, or CDK) creates the stack build, importing libraries and other elements. Depending on how the infrastructure tool works and the delivery workflow used by the team, the stack artifact could be files generated in a working directory, a branch or tag in source control, or an archive like a ZIP file stored in a repository.

The stack tool compiles the code build, along with any instance-specific information like configuration parameters, to create the stack model, the desired state for the infrastructure resources. The tool then executes the stack model by calling the IaaS API to create, modify, or destroy resources to make the infrastructure stack instance match the desired state. The stack instance is the live infrastructure available for use by workloads. See “Build Stage: Preparing for Distribution” for workflow approaches to building and deploying stacks.

Chapter 7 describes design patterns for stacks. “Reusable Stack” defines one of these patterns, in which a single infrastructure stack project may be reused to deploy multiple stack instances. Chapter 8 then describes patterns for configuring stack instances when deploying them.

“Stack” as a Term

Not all infrastructure tools use the term “stack” to describe a deployable unit of resources. CloudFormation and Pulumi both use the term this way, but HashiCorp has much more recently introduced a composition feature that, confusingly, is named “stack.”

Terraform calls a separately deployable unit within a stack a “component.” Outside of a Terraform stack (composition), HashiCorp doesn’t have a separate term for the concept, although you sometimes see the terms “project,” “configuration,” or “workspace” used in a related way.

In this book, I describe patterns and practices that should be relevant to any tool. The concept of a deployable infrastructure component is one of the most common concepts in the book. Because “deployable infrastructure component” is too awkward, I have been using the word stack for this concept since the first edition of this book in 2016. It is the most widely used term for this in the industry, even if it isn’t universal.

I hope that readers will be able to map the patterns and practices described for stacks in this book to whatever tool they use. When I refer to Terraform’s stack feature, I will generally say “Terraform stack (composition).” Otherwise, when I use the term “stack” by itself, it follows my definition.

Infrastructure Code Libraries

An infrastructure code library is a component that groups infrastructure code so that it can be shared and reused across stack projects. Common implementations include Terraform modules, CDK Level 3 constructs, CloudFormation modules, and Pulumi component resources.

Figure 6-5 shows where code libraries like modules fit into the infrastructure code deployment process.

As described earlier, the infrastructure tool uses the stack project code to prepare a stack build in the Assembly step, which includes resolving and retrieving libraries. In some cases, the library code is stored and edited together with the stack project code. In others, libraries are imported from a source code repository or an artifact repository like a Terraform / OpenTofu module registry.

iac3 0605
Figure 6-5. Where libraries fit in the infrastructure code deployment process

Chapter 10 covers patterns and antipatterns for using code libraries to build stacks.

Libraries as Deployable Stacks

One of the patterns defined in Chapter 10 is worth mentioning here as part of this chapter’s discussion of types of infrastructure components. The Stack Module pattern (see “Stack Module”) uses a code library to implement a stack. People do this is because some popular infrastructure tools don’t have support for an independent deployment-oriented component but do have good support for libraries.1 Specifically, Terraform and OpenTofu have support for modules as components that can be versioned, distributed, shared, and reused. Infrastructure is deployed using code in a project code folder that may import versioned modules but that doesn’t have the attributes of a component.

A stack module implements a complete, deployable stack (perhaps even importing other libraries) as a library, using the infrastructure tool’s library support. To deploy an instance of the stack, an infrastructure code project is created that imports the stack module. A stack project that exists only to import and deploy a stack module is a wrapper stack (see “Deployment Wrapper Stack”).

Tools such as Terragrunt are designed to help teams implement wrapper stacks. HCP Terraform’s no-code provisioning feature dynamically generates a deployment wrapper project for a stack module.

Sharing and Reuse of Infrastructure Code

Sharing and reusing infrastructure code is an obvious way to reduce the surface area of code and systems that need to be maintained and updated, and to get more value out of the work we put into building and managing the code. Trade-offs and pitfalls exist, of course. Often an existing component doesn’t quite meet the needs of a new use case, so we need to exercise judgment on whether we can modify that component to meet the new requirements while still meeting the old, or whether we should create a new component.

Of the infrastructure components introduced in this chapter, code libraries are, generally, designed to be shared and reused. Infrastructure stacks can usually be treated this way, although not all infrastructure tools are designed to support stacks as a shareable component. Infrastructure compositions, as mentioned, are a newer concept and are less mature. Not all of them are designed as reusable components.

Sharing Infrastructure Code Components

The don’t repeat yourself (DRY) principle says, “Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.”2 If you copy the same code to use in multiple places, then discover the need to make a change, it can be difficult to track down all the places to make that change.

Code libraries are a common solution for reducing duplicated code across infrastructure stack projects. However, sharing a library across stacks is a trade-off between reuse and coupling. Making a change to a library impacts all the stacks that use it. A simple change may require all the library’s users to retest their components and systems to make sure the library doesn’t break something. A larger, breaking change needed to adapt the library to support a new project might force changes to all the stacks that use it.

The DRY principle is best viewed as applying not to specific code but to higher-level abstractions. For example, one team created a Terraform module to replace all references to Amazon Elastic Compute Cloud (EC2) instances. The team noticed that the code to define an EC2 instance, used in multiple projects, all looked pretty much the same, so decided it needed to be made DRY.

However, once the team members implemented a virtual_server module to replace the uses of the raw EC2 resource declarations, they noticed that the references to their modules didn’t look any more DRY than the original code. They realized that their codebase defined virtual servers for multiple purposes, including web servers, several types of application servers, and some servers that hosted third-party software packages. Because these servers needed to be configured differently, their virtual_server module needed to offer most of the configuration options from the underlying EC2 resource definition. So their module was a thin wrapper over the native Terraform resource. It didn’t add any value but did add complexity to their codebase.

After a rethink, the team members created a java_application_server module. They had multiple places in their codebase where they defined these virtual servers, and there were only a few differences in each of these places. They defined their web servers in only one location, so a module would be a waste for that. Each of the third-party packages that they hosted was different enough that sharing code provided no benefit, so they defined these servers simply, directly using the Terraform resource definition.3

Sharing Stack Code Across Multiple Instances

Infrastructure stacks are the fundamental deployable component. Deployment creates the opportunity to share and reuse a stack project by using it to provision multiple stack instances.

The example infrastructure resource design illustrated earlier in this chapter (Figure 6-2) showed two software services running in the FoodSpin system: the menu service and the ordering service. Each service has a database instance. Figure 6-6 shows how a single code project can be used to deploy a separate database stack instance for each of these services.

iac3 0606
Figure 6-6. A single stack project deployed to multiple stack instances

The FoodSpin team builds the database service stack once every time it’s changed, creating a new package version. This stack package is then deployed twice, once for each of the services that needs an instance. The teams may need their databases to be configured differently. The menu service may need a larger database than the ordering service, or the search service database may need to be tuned differently than the others. These variations can be managed by making the shared stack configuration. Chapter 8 describes approaches for implementing stack configuration.

In this example, multiple instances of a stack are deployed in a single environment. It’s also common to reuse a stack project across multiple environments, like environments used to develop and test software. Chapter 7 defines sharing stack code in this way as the Reusable Stack pattern (“Reusable Stack”), which is a building block used extensively throughout this book.

Sharing Stack Instances Across Workloads

We can share an infrastructure code library across multiple stack projects, and we can share a stack project across multiple stack instances. We can also share a stack instance across multiple workloads. Figure 6-7 shows another subset of the infrastructure from the example FoodSpin infrastructure design.

iac3 0607
Figure 6-7. A stack instance shared by multiple workloads

In addition to the two database stack instances, each dedicated to one of the FoodSpin software services, this diagram shows a single shared network stack instance, used by both of the software services. This could be called a shared tenancy instance, and the database instances could be called single tenancy instances. However, the term “tenancy” is often used to refer to whether different customers share a single deployed software instance, so I’ll generally use the terms dedicated instance and shared instance.

Application-Driven Infrastructure Design

Infrastructure is sometimes designed in complete detachment from the applications and services it enables. Infrastructure as Code creates an opportunity to rethink this siloed mindset, aligning infrastructure more closely with its workloads. The traditional approach leads to inflexible horizontal layers.

The field of software architecture has largely moved from horizontal to vertical designs, structuring software around the services it provides to users rather than tiers built around specific technologies. Let’s consider how these two approaches apply to infrastructure design, and then define a workflow for designing infrastructure that uses the component model introduced in this chapter to align infrastructure with applications.

Horizontal Design

Traditionally, architects organized systems functionally: networking stuff together, database stuff together, and OS stuff together. Figure 6-8 shows three infrastructure stacks organized around horizontal concerns. Each stack provides resources used by four FoodSpin software services: one for compute resources, one for databases, and the third for networking.

iac3 0608
Figure 6-8. Infrastructure grouped into horizontal layers

Note that these infrastructure stacks don’t necessarily duplicate code or components. For example, the database infrastructure stack may use a single database code library to provision each of the four database instances. The issue with the horizontal infrastructure architecture isn’t duplication; it’s scope and ownership of change.

Making a configuration change to the database instance for the menu service changes requires editing the database stack that is shared with the other workloads. So the scope of risk for a change to one instance is all the instances defined in the stack. People need to spend extra effort to coordinate work across the teams, in case more than one team is making changes to the stack at the same time.

A common solution to this problem is to have a central team, such as a database team, own the shared database infrastructure composition. This disempowers the teams that own the services, requiring them to raise a request to the database team for even a small change, and makes the capacity of the database team a bottleneck for all the services.

Vertical Design

An alternative is to organize the infrastructure code to align with workloads, as shown in Figure 6-9. In this example, the infrastructure resources for each software service are provisioned in dedicated stacks.

iac3 0609
Figure 6-9. Infrastructure grouped into vertical layers

As with the previous example, the infrastructure stacks may be defined using shared libraries, so that the code to define database instances, for example, is not duplicated. If the infrastructure for each service is very similar, the team can create a single, configurable shared stack project as described earlier in this chapter. In any case, having separately deployed stack instances for each service means one team can change its infrastructure’s configuration without needing to coordinate with other teams and without worrying about accidentally breaking another team’s infrastructure.

Shared Infrastructure Included in Vertical Design

Earlier in this chapter, I described sharing deployed infrastructure instances among workloads. Unlike the examples of horizontal layers that included infrastructure dedicated to multiple workloads, a shared infrastructure stack instance is usually needed in these cases. However, the shared infrastructure stack can be limited in scope, designed around a cohesive service, and deployed separately from other stack instances that contain workload-specific infrastructure. Figure 6-10 shows a design with some workload-specific infrastructure stacks and some shared stacks.

iac3 0610
Figure 6-10. Mixing vertical components with shared components

The workload-specific infrastructure stacks define a database instance and service-specific network elements, such as firewall and load-balancer rules for traffic to that service. Two infrastructure stacks are shared, one that defines a compute cluster, the other defining shared network resources like a VPC and subnets.

Reference Application-Driven Infrastructure Design

Supporting multiple environments adds another dimension for composing infrastructure. Figure 6-11 provides an example of designing infrastructure across multiple environments with an application-driven approach.

The design has multiple levels, each of which may be implemented as one or more infrastructure stacks or compositions. It starts with dedicated infrastructure provisioned specifically for a given workload.

iac3 0611
Figure 6-11. Reference infrastructure design layers

The next level is shared infrastructure components that may be used by multiple workloads and workload infrastructure components, like the shared network and compute stacks described earlier.

Environment infrastructure would include components needed for the environment as a whole, such as network routing policies. Global infrastructure would be used across all environments, often focusing on common controls and governance policies. Chapter 12 discusses designing and implementing environments and IaaS resources groups and accounts.

The following are two useful design principles for deciding where to implement infrastructure resources and components within the overall structure shown here:

Implement infrastructure at the most specific relevant level

Ideally, components at lower levels of infrastructure, such as an environment, should be minimal, including only resources that are actually used by all the higher levels of infrastructure and workloads it supports. For example, load-balancing rules for a specific workload should be defined in the dedicated infrastructure for that service, not in the environment.

Lower levels of infrastructure should not know about higher levels of infrastructure

For example, an environment should not include configuration for specific workloads or shared services. Workloads and infrastructure components provisioned at higher levels are consumers of lower-level infrastructure. So environment infrastructure is a provider to shared infrastructure and to workload infrastructure. A provider should not be implemented with knowledge of its consumers, to avoid circular relationships.

Design Workflow

An application-driven design approach to infrastructure starts by understanding the workloads it will run and working backward. This leads to a natural design flow based on following the infrastructure code deployment process backward. Start with the runtime context, which is the infrastructure as it is used by workloads, then consider how it will be deployed, and work back to how to organize the code so that people and teams can work on it. Figure 6-12 shows this workflow.

iac3 0612
Figure 6-12. Application-driven design workflow

Step 1 starts with the workload and uses that to design the appropriate infrastructure compositions in step 2. Step 3 considers how the infrastructure might be best organized as deployable stacks. Step 4 moves into the realm of code, defining the projects needed to create the deployment stacks. The codebase may make use of libraries, as shown in step 5. Step 6 is writing the infrastructure code that defines the specific IaaS resources that will be produced at runtime.

The component model described in this chapter, and the application-driven design workflow to implement it, may be useful for larger, more complex systems. However, the starting point for implementing Infrastructure as Code will normally be either a small system or a small subset of a larger system. The system may evolve to need all the moving parts described here, but this should be done organically, driven by immediate needs and what the team learns along the way, rather than by speculating about future needs.

Conclusion

The last few chapters have explored how infrastructure code works, guidance for designing infrastructure, and, in this chapter, infrastructure components. The components described in this chapter will be used throughout the rest of the book to explain patterns for testing, delivering, and managing infrastructure by using code. The terms used here—infrastructure compositions, infrastructure deployment stacks, and infrastructure code libraries—are not used universally across tool vendors. However, the concepts apply to whatever tools you may use, so we need consistent terminology to describe them in this book.

The rest of the chapters in this part of the book dive into approaches to designing and implementing infrastructure by using the component types described in this chapter. We’ll start with a more detailed look at infrastructure deployment stacks.

1 HashiCorp released support for Terraform stack (composition) components as this book is being finalized. It will be interesting to see how this affects the use of modules as distributable stack artifacts.

2 The DRY principle can be found in The Pragmatic Programmer: Your Journey to Mastery by David Thomas and Andrew Hunt (Addison-Wesley).

3 I recommend Sandi Metz’s post, “The Wrong Abstraction”. Kent C. Dodds’s post “AHA Programming” builds on Metz’s post and on Cher Scarlett’s observation that one should “Avoid Hasty Abstractions.”