Small Modules

Imagine you were using a general-purpose programming language such as Java or Python or Ruby, and you came across a single, massive function that was 20,000 lines long:

def huge_function(data_set)
  x_pca = PCA(n_components=2).fit_transform(X_train)
  clusters = clf.fit_predict(X_train)
  ax = plt.subplots(1, 2, figsize=(4))
  ay = plt.subplots(0, 2, figsize=(2))
  fig = plt.subplots(3, 4, figsize=(5))
  fig.subplots_adjust(top=0.85)

  predicted = svc_model.predict(X_test)
  images_and_predictions = list(zip(images_test, predicted))

  for x in 0..xlimit
    ax[0].scatter(X_pca[x], X_pca[1], c=clusters)
    ax[0].set_title('Predicted Training Labels')
    ax[1].scatter(X_pca[x], X_pca[1], c=y_train)
    ax[1].set_title('Actual Training Labels')
    ax[2].scatter(X_pca[x], X_pca[1], c=clusters)
  end

  for y in 0..ylimit
    ay[0].scatter(X_pca[y], X_pca[1], c=clusters)
    ay[0].set_title('Predicted Training Labels')
    ay[1].scatter(X_pca[y], X_pca[1], c=y_train)
    ay[1].set_title('Actual Training Labels')
    ay[2].scatter(X_pca[y], X_pca[1], c=clusters)
  end

  #
  # ... 20,000 more lines...
  #
end

You immediately know this is a code smell and that the better approach is to refactor it into a number of small, standalone functions that each do one thing:

def calculate_images_and_predictions(images_test, predicted)
  x_pca = PCA(n_components=2).fit_transform(X_train)
  clusters = clf.fit_predict(X_train)
  ax = plt.subplots(1, 2, figsize=(4))
  fig = plt.subplots(3, 4, figsize=(5))
  fig.subplots_adjust(top=0.85)

  predicted = svc_model.predict(X_test)
  return list(zip(images_test, predicted))
end

def process_x_coords(ax)
  for x in 0..xlimit
    ax[0].scatter(X_pca[x], X_pca[1], c=clusters)
    ax[0].set_title('Predicted Training Labels')
    ax[1].scatter(X_pca[x], X_pca[1], c=y_train)
    ax[1].set_title('Actual Training Labels')
    ax[2].scatter(X_pca[x], X_pca[1], c=clusters)
  end

  return ax
end

def process_y_coords(ax)
  for y in 0..ylimit
    ay[0].scatter(X_pca[y], X_pca[1], c=clusters)
    ay[0].set_title('Predicted Training Labels')
    ay[1].scatter(X_pca[y], X_pca[1], c=y_train)
    ay[1].set_title('Actual Training Labels')
    ay[2].scatter(X_pca[y], X_pca[1], c=clusters)
  end

  return ay
end

#
# ... Lots more small functions...
#

Let’s refactor the code accordingly into three smaller modules:

modules/cluster/asg-rolling-deploy

A generic, reusable, standalone module for deploying an ASG that can do a zero-downtime, rolling deployment.

modules/networking/alb

A generic, reusable, standalone module for deploying an ALB.

modules/services/hello-world-app

A module specifically for deploying the “Hello, World” app.

Before getting started, make sure to run terraform destroy on any webserver-cluster deployments you have from previous chapters. After you do that, you can start putting together the asg-rolling-deploy and alb modules. Create a new folder at modules/cluster/asg-rolling-deploy and move (i.e., cut and paste) the following resources from module/services/webserver-cluster/main.tf to modules/cluster/asg-rolling-deploy/main.tf:

• aws_launch_configuration

• aws_autoscaling_group

• aws_autoscaling_schedule (both of them)

• aws_security_group (for the Instances, but not for the ALB)

• aws_security_group_rule (both of the rules for the Instances, but not those for the ALB)

• aws_cloudwatch_metric_alarm (both of them)

Next, move the following variables from to module/services/webserver-cluster/variables.tf to modules/cluster/asg-rolling-deploy/variables.tf:

• cluster_name

• ami

• instance_type

• min_size

• max_size

• enable_autoscaling

• custom_tags

• server_port

• aws_lb

• aws_lb_listener

• aws_security_group (the one for the ALB, but not for the Instances)

• aws_security_group_rule (both of the rules for the ALB, but not for the Instances)

Create modules/networking/alb/variables.tf and define a single variable within:

variable "alb_name" {
  description = "The name to use for this ALB"
  type        = string
}

Use this variable as the name argument of the aws_lb resource:

resource "aws_lb" "example" {
  name               = var.alb_name
  load_balancer_type = "application"
  subnets            = data.aws_subnet_ids.default.ids
  security_groups    = [aws_security_group.alb.id]
}

And the name argument of the aws_security_group resource:

resource "aws_security_group" "alb" {
  name = var.alb_name
}

# Simple function to do addition
def add(x, y)
  return x + y
end

# Simple function to do subtraction
def sub(x, y)
  return x - y
end

# Simple function to do multiplication
def multiply(x, y)
  return x * y
end

You can use function composition to put them together by taking the outputs from add and sub and passing them as the inputs to multiply:

# Complex function that composes several simpler functions
def do_calculation(x, y)
  return multiply(add(x, y), sub(x, y))
end

Open up modules/cluster/asg-rolling-deploy/variables.tf and add four new input variables:

variable "subnet_ids" {
  description = "The subnet IDs to deploy to"
  type        = list(string)
}

variable "target_group_arns" {
  description = "The ARNs of ELB target groups in which to register Instances"
  type        = list(string)
  default     = []
}

variable "health_check_type" {
  description = "The type of health check to perform. Must be one of: EC2, ELB."
  type        = string
  default     = "EC2"
}

variable "user_data" {
  description = "The User Data script to run in each Instance at boot"
  type        = string
  default     = ""
}

The first variable, subnet_ids, directs the asg-rolling-deploy module to the subnets to deploy into. Whereas the webserver-cluster module was hardcoded to deploy into the Default VPC and subnets, by exposing the subnet_ids variable, you allow this module to be used with any VPC or subnets. The next two variables, target_group_arns and health_check_type, configure how the ASG integrates with load balancers. Whereas the webserver-cluster module had a built-in ALB, the asg-rolling-deploy module is meant to be a generic module, so exposing the load-balancer settings as input variables allows you to use the ASG with a wide variety of use cases; e.g., no load balancer, one ALB, multiple NLBs, and so on.

Take these three input variables and pass them through to the aws_autoscaling_group resource in modules/cluster/asg-rolling-deploy/main.tf, replacing the previously hardcoded settings that are referencing resources (e.g., the ALB) and data sources (e.g., aws_subnet_ids) that we did not bother copying into the asg-rolling-deploy module:

resource "aws_autoscaling_group" "example" {
  # Explicitly depend on the launch configuration's name so each time
  # it's replaced, this ASG is also replaced
  name = "${var.cluster_name}-${aws_launch_configuration.example.name}"

  launch_configuration = aws_launch_configuration.example.name
  vpc_zone_identifier  = var.subnet_ids

  # Configure integrations with a load balancer
  target_group_arns    = var.target_group_arns
  health_check_type    = var.health_check_type

  min_size = var.min_size
  max_size = var.max_size

  # Wait for at least this many instances to pass health checks before
  # considering the ASG deployment complete
  min_elb_capacity = var.min_size

  # (...)
}

resource "aws_launch_configuration" "example" {
  image_id        = var.ami
  instance_type   = var.instance_type
  security_groups = [aws_security_group.instance.id]
  user_data       = var.user_data

  # Required when using a launch configuration with an auto scaling group.
  # https://www.terraform.io/docs/providers/aws/r/launch_configuration.html
  lifecycle {
    create_before_destroy = true
  }
}

output "asg_name" {
  value       = aws_autoscaling_group.example.name
  description = "The name of the Auto Scaling Group"
}

output "instance_security_group_id" {
  value       = aws_security_group.instance.id
  description = "The ID of the EC2 Instance Security Group"
}

output "alb_dns_name" {
  value       = aws_lb.example.dns_name
  description = "The domain name of the load balancer"
}

output "alb_http_listener_arn" {
  value       = aws_lb_listener.http.arn
  description = "The ARN of the HTTP listener"
}

output "alb_security_group_id" {
  value       = aws_security_group.alb.id
  description = "The ALB Security Group ID"
}