Cloud Native and Application-Driven Infrastructure

Cloud native software is designed and implemented to exploit the dynamic nature of modern infrastructure. Unlike older-generation software, instances of a cloud native application can be transparently added, removed, and shifted across underlying infrastructure. The underlying platform dynamically allocates compute and storage resources and routes traffic to and from the application. The application integrates seamlessly with services like monitoring, logging, authentication, and encryption.

The folks at Heroku articulated the twelve-factor methodology for building applications to run on cloud infrastructure. Cloud native, as a phrase, is often associated with the Kubernetes ecosystem.1

Many organizations have existing software portfolios that aren’t cloud native. While they may convert or rewrite some of their software to become cloud native, in many cases the cost of doing so isn’t justified by the benefits. An application-driven infrastructure strategy involves building application runtime environments for applications using modern, dynamic infrastructure.

Teams provide application runtimes for new applications that run as serverless code or in containers. They also provide infrastructure to support existing applications. All of the infrastructure is defined, provisioned, and managed as code. Application-driven infrastructure may be provided dynamically using an abstraction layer (see “Building an Abstraction Layer”).

Application Runtime Targets

Implementing an application-driven strategy starts with analyzing the runtime requirements of your application portfolio. You then design application runtime solutions to meet those requirements, implementing reusable stacks, stack components, and other elements teams can use to assemble environments for specific applications.

Deploying Applications to Servers

Servers, whether physical or virtual, are the traditional runtime platform. An application is packaged using an operating system packaging format such as an RPM, a .deb file, or a Windows MSI. Or it is packaged in a language runtime format, such as a Ruby gem or Java .war file. More recently, container images, such as Docker images, have gained popularity as a format for packaging and deploying applications to servers.

Defining and provisioning servers as code is the topic of Chapter 11. This topic overlaps with application deployment, given the need to decide when and how to run deployment commands (see “Configuring a New Server Instance”).

Packaging Applications in Containers

Containers pull dependencies from the operating system into an application package, the container image, as shown in Figure 10-2.2

Figure 10-2. Dependencies may be installed on the host OS or bundled in containers

Including dependencies in the container makes it larger than typical operating system or language packages, but has several advantages:

• A container creates a more consistent environment for running an application. Without a container, the application relies on libraries, configuration, user accounts, and other elements that may be different on different servers. A container bundles the runtime environment along with the application and its dependencies.

• A containerized application is mostly isolated from the server it runs on, which gives you flexibility for where you can run it.

• By packaging the application’s operating system context in the container image, you simplify and standardize the requirements for the host server. The server needs to have the container execution tooling installed, but little else.

• Reducing the variability of runtime environments for an application improves quality assurance. When you test a container instance in one environment, you can be reasonably confident it will behave the same in other environments.

Deploying Applications to Server Clusters

People have been deploying applications to groups of servers since before container-based application clusters were a thing. The usual model is to have a server cluster (as described in “Compute Resources”), and run an identical set of applications on each server. You package your applications the same way you would for a single server, repeating the deployment process for each server in the pool, perhaps using a remote command scripting tool like Capistrano or Fabric. See Figure 10-3.

Figure 10-3. Applications are deployed to each server in a cluster

If you deploy an application to multiple servers, you need to decide how to orchestrate the deployment. Do you deploy the application to all of the servers at once? Do you need to take the entire service offline while you do this? Or do you upgrade one server at a time? You can leverage incremental deployment to servers for progressive deployment strategies like the blue-green and canary deployment patterns (see “Changing Live Infrastructure” for more on these strategies).

In addition to deploying application code onto servers, you may need to deploy other elements like changes to data structures or connectivity.

Deploying Applications to Application Clusters

As discussed in “Compute Resources”, an application hosting cluster is a pool of servers that runs one or more applications. Unlike a server cluster, where each server runs the same set of applications, different servers in an application cluster may run different groups of application instances (see Figure 10-4).

When you deploy an application to the cluster, a scheduler decides which host servers to run instances of the application on. The schedule may change this distribution, adding and removing application instances across host servers according to different algorithms and settings.

Figure 10-4. Applications are deployed to the cluster and distributed across host nodes

In the old days,3 the most popular application clusters were Java-based (Tomcat, Websphere, Weblogic, JBoss, and others). A wave of cluster management systems emerged a few years ago, including Apache Mesos and DC/OS, many inspired by Google’s Borg.⁴ Systems focused on orchestrating container instances have overwhelmed application servers and cluster orchestrators in more recent years.

Defining and provisioning clusters as code is the topic of Chapter 14. Once you have your cluster, it may be simple to deploy a single application onto it. Package the application with Docker and push it to the cluster. But more complex applications, with more moving parts, have more complex requirements.

Packages for Deploying Applications to Clusters

Modern applications often involve multiple processes and components deployed across complex infrastructure. A runtime environment needs to know how to run these various pieces:

Different runtime platforms provide different functionality, and many have their own packaging and configuration format. Often, these platforms use a deployment manifest that refers to the actual deployment artifacts (for example, a container image), rather than an archive file that includes all of the deployable pieces. Examples of cluster application deployment manifests include:

• Helm charts for Kubernetes clusters

• Weave Cloud Kubernetes deployment manifests

• AWS ECS Services, which you can define as code using your favorite stack management tool

• Azure App Service Plans

• CNAB Cloud Native Application Bundle

Different deployment manifests and packages work at different levels. Some are focused on a single deployable unit, so you need a manifest for each application. Some define a collection of deployable services. Depending on the tool, each of the services within the collection may have a separate manifest, with a higher-level manifest defining common elements and integration parameters.

A manifest for the ShopSpinner web server deployment might look like the pseudocode shown in Example 10-1.

service:
  name: webservers
  organization: ShopSpinner
  version: 1.0.3
application:
  name: nginx
  container_image:
    repository: containers.shopspinner.xyz
    path: /images/nginx
    tag: 1.0.3
  instance:
    count_min: 3
    count_max: 10
    health_port: 443
    health_url: /alive
  connectivity:
    inbound:
      id: https_inbound
      port: 443
      allow_from: $PUBLIC_INTERNET
      ssl_cert: $SHOPSPINNER_PUBLIC_SSL_CERT
    outbound:
      port: 443
      allow_to: [ $APPSERVER_APPLICATIONS.https_inbound ]

This example specifies where and how to find the container image (the container_image block), how many instances to run, and how to check their health. It also defines inbound and outbound connection rules.

Deploying FaaS Serverless Applications

In Chapter 3, I listed some FaaS serverless application runtime platforms as a type of compute resource (see “Compute Resources”). Most of these have their own format for defining application runtime requirements, and packaging these with code and related pieces for deploying to an instance of the runtime.

When writing a FaaS application, the details of whatever servers or containers your code runs in are hidden from you. But your code probably needs infrastructure to work. For example, you may need network routing for inbound or outbound connections, storage, or message queues. Your FaaS framework may integrate with the underlying infrastructure platform, automatically provisioning the necessary infrastructure. Or you may need to define the infrastructure elements in a separate stack definition tool. Many stack tools like Terraform and CloudFormation let you declare your FaaS code provisioning as a part of an infrastructure stack.

Chapter 14 is relevant for defining and provisioning FaaS runtimes to run your code.

Data is often an afterthought for deploying and running applications. We provision databases and storage volumes but, as with many parts of infrastructure, it’s making changes to them that turns out to be hard. Changing data and structures is time consuming, messy, and risky.

Application deployment often involves creating or changing data structures, including converting existing data when structures change. Updating data structures should be a concern of the application and application deployment process, rather than the infrastructure platform and runtime. However, the infrastructure and application runtime services need to support maintaining data when infrastructure and other underlying resources change or fail. See “Data Continuity in a Changing System” for ways to approach this challenge.

application:
  name: db_cluster
  compute_instance:
    memory: 2 GB
    container_image: db_cluster_node_application
  storage_volume:
    size: 50 GB
    volume_image: db_cluster_node_volume

In addition to compute resources to execute code and storage resources to hold data, applications need networking for inbound and outbound connectivity. A server-oriented application package, such as for a web server, might configure network ports and add encryption keys for incoming connections. But traditionally, they have relied on someone to configure infrastructure outside the server separately.

You can see this in part of Example 10-1, shown here:

application:
  name: nginx
  connectivity:
    inbound:
      id: https_inbound
      port: 443
      allow_from: $PUBLIC_INTERNET
      ssl_cert: $SHOPSPINNER_PUBLIC_SSL_CERT
    outbound:
      port: 443
      allow_to: [ $APPSERVER_APPLICATIONS.https_inbound ]

This example defines inbound and outbound connections, referencing other parts of the system. These are the public internet, presumably a gateway, and inbound HTTPS ports for application servers, defined and deployed to the same cluster using their own deployment manifests.

Application runtimes provide many common services to applications. Many of these services are types of service discovery.

Service Discovery

Applications and services running in an infrastructure often need to know how to find other applications and services. For example, a frontend web application may send requests to a backend service to process transactions for users.

Hardcoded IP addresses

Allocate IP addresses for each service. For example, the monitoring server always runs on 192.168.1.5. If you need to change the address or run multiple instances of a service (for example, for a controlled rollout of a major upgrade), you need to rebuild and redeploy applications.

Hostfile entries

Use server configuration to generate the /etc/hosts file (or equivalent) on each server, mapping service names to the current IP address. This method is a messier alternative to DNS, but I’ve seen it used to work around legacy DNS implementations.7

DNS (Domain Name System)

Use DNS entries to map service names to their current IP address, either using DNS entries managed by code, or DDNS (Dynamic DNS). DNS is a mature, well-supported solution to the problem.

Resource tags

Infrastructure resources are tagged to indicate what services they provide, and the context such as environments. Discovery involves using the platform API to look up resources with relevant tags. Care should be taken to avoid coupling application code to the infrastructure platform.

Configuration registry

Application instances can maintain the current connectivity details in a centralized registry (see “Configuration Registry”), so other applications can look it up. This may be helpful when you need more information than the address; for example, health or other status indications.

Sidecar

A separate process runs alongside each application instance. The application may use the sidecar as a proxy for outbound connections, a gateway for inbound connections, or as a lookup service. The sidecars will need some method to do network discovery itself. This method could be one of the other discovery mechanisms, or it might use a different communication protocol.8 Sidecars are usually part of a service mesh ( I discuss service meshes in “Service Mesh”), and often provide more than service discovery. For example, a sidecar may handle authentication, encryption, logging, and monitoring.

API gateway

An API gateway is a centralized HTTP service that defines routes and endpoints. It usually provides more services than this; for example, authentication, encryption, logging, and monitoring. In other words, an API gateway is not unlike a sidecar, but is centralized rather than distributed.9

The purpose of infrastructure is to run useful applications and services. The guidance and ideas in this book should help you in providing collections of infrastructure resources in the shapes needed to do this. An application-driven approach to infrastructure focuses on the runtime requirements of applications, helping you to design stacks, servers, clusters, and other middle-tier constructs for running applications.