This chapter covers

• Choosing a project distribution method
• Using hosted infrastructure
• Running and using your own registry
• Understanding manual image distribution workflows
• Distributing image sources

You have your own images of software you’ve written, customized, or just pulled from the internet. But what good is an image if nobody can install it? Docker is different from other container management tools because it provides image distribution features.

There are several ways to get your images out to the world. This chapter explores those distribution paradigms and provides a framework for making or choosing one or more for your own projects.

The most difficult thing about choosing a distribution method is choosing the appropriate method for your situation. To help with this problem, each method presented in this chapter is examined on the same set of selection criteria.

The first thing to recognize about distributing software with Docker is that there’s no universal solution. Distribution requirements vary for many reasons, and several methods are available. Every method has Docker tools at its core, so it’s always possible to migrate from one to another with minimal effort. The best way to start is by examining the full spectrum of options at a high level.

The methods included in the spectrum range from hosted registries such as Docker Hub to totally custom distribution architectures or source-distribution methods. We cover some of these subjects in more detail than others. We also place particular focus on private registries because they provide the most balance between the two concerns.

Having a spectrum of choices illustrates your range of options, but you need a consistent set of selection criteria in order to determine which you should use.

The following identified selection criteria are based on differences across the spectrum and on common business concerns. When making a decision, consider how important each of these is in your situation:

• Cost
• Visibility
• Transport speed or bandwidth overhead
• Longevity control
• Availability control
• Access control
• Artifact integrity
• Artifact confidentiality
• Requisite expertise

How each distribution method stacks up against these criteria is covered in the relevant sections over the rest of this chapter.

Cost is the most obvious criterion, and the distribution spectrum ranges in cost from free to very expensive, and “it’s complicated.” Lower cost is generally better, but cost is typically the most flexible criterion. For example, most people will trade cost for artifact confidentiality if the situation calls for it.

Visibility is the next most obvious criterion for a distribution method. Secret projects or internal tools should be difficult if not impossible for unauthorized people to discover. In another case, public works or open source projects should be as visible as possible to promote adoption.

Transportation speed and bandwidth overhead are the next most flexible criteria. File sizes and image installation speed will vary between methods that leverage image layers, concurrent downloads, and prebuilt images and those that use flat image files or rely on deployment-time image builds. High transportation speeds or low installation latency is critical for systems that use just-in-time deployment to service synchronous requests. The opposite is true in development environments or asynchronous processing systems.

Availability control is the ability to control the resolution of availability issues with your repositories. Hosted solutions provide no availability control. Businesses typically provide a service-level agreement (SLA) on availability if you’re a paying customer, but there’s nothing you can do to directly resolve an issue. On the other end of the spectrum, private registries or custom solutions put both the control and responsibility in your hands.

Access control protects your images from modification or access by unauthorized parties. Varying degrees of access control are available. Some systems provide only access control of modifications to a specific repository, whereas others provide course control of entire registries. Still other systems may include pay walls or digital rights management controls. Projects typically have specific access-control needs dictated by the product or business. This makes access-control requirements one of the least flexible and most important to consider.

Artifact integrity and confidentiality both fall in the less-flexible and more-technical end of the spectrum. Artifact integrity is trustworthiness and consistency of your files and images. Violations of integrity may include man-in-the-middle attacks, in which an attacker intercepts your image downloads and replaces the content with their own. They might also include malicious or hacked registries that lie about the payloads they return.

Artifact confidentiality is a common requirement for companies developing trade secrets or proprietary software. For example, if you use Docker to distribute cryptographic material, confidentiality will be a major concern. Artifact integrity and confidentiality features vary across the spectrum. Overall, the out-of-the-box distribution security features won’t provide the tightest confidentiality or integrity. If that’s one of your needs, an information security professional will need to implement and review a solution.

With this strong set of selection criteria, you can begin learning about and evaluating various distribution methods. The following sections evaluate these methods against the criteria by using ratings of Worst, Bad, Good, Better, and Best. The best place to start is on the far left of the spectrum with hosted registries.

As a reminder, Docker registries are services that make repositories accessible to Docker pull commands. A registry hosts repositories. The simplest way to distribute your images is by using hosted registries.

The example images used in this book are distributed with public repositories hosted on Docker Hub and Quay.io. By the end of this section, you’ll understand how to publish your own images by using hosted registries and how hosted registries measure up to the selection criteria.

Once you have your account, you need to create an image to publish. Create a new Dockerfile named HelloWorld.df and add the following instructions:

FROM busybox:latest          1
CMD echo 'Hello World!'

• 1 From HelloWorld.df

The push refers to a repository
[dockerinaction/hello-dockerfile] (len: 1)
7f6d4eb1f937: Image already exists
8c2e06607696: Image successfully pushed
6ce2e90b0bc7: Image successfully pushed
cf2616975b4a: Image successfully pushed
Digest:
  sha256:ef18de4b0ddf9ebd1cf5805fae1743181cbf3642f942cae8de7c5d4e375b1f20

The command output includes upload statuses and the resulting repository content digest. The push operation will create the repository on the remote registry, upload each of the new layers, and then create the appropriate tags.

Your public repository will be available to the world as soon as the push operation is completed. Verify that this is the case by searching for your username and your new repository. For example, use the following command to find the example owned by the dockerinaction user:

docker search dockerinaction

Replace the dockerinaction username with your own to find your new repository on Docker Hub. You can also log in to the Docker Hub website and view your repositories to find and modify your new repository.

Public repositories on hosted registries are the best choice for owners of open source projects or people who are just getting started with Docker. People should still be skeptical of software that they download and run from the internet, so public repositories that don’t expose their sources can be difficult for some users to trust. Hosted (trusted) builds solve this problem to a certain extent.

Private repositories are similar to public repositories from an operational and product perspective. Most registry providers offer both options, and any differences in provisioning through their websites will be minimal. Because the Docker registry API makes no distinction between the two types of repositories, registry providers that offer both generally require you to provision private registries through their website, app, or API.

The following commands prompt you to authenticate with the registries provided by Docker Hub and Quay.io. After creating accounts and authenticating, you’ll have full access to your public and private repositories on all three registries. The login subcommand takes an optional server argument:

docker login
# Username: dockerinaction
# Password:
# Email: book@dockerinaction.com
# WARNING: login credentials saved in /Users/xxx/.dockercfg.
# Login Succeeded

docker login quay.io
# Username: dockerinaction
# Password:
# Email: book@dockerinaction.com
# WARNING: login credentials saved in /Users/xxx/.dockercfg.
# Login Succeeded

Individuals and small teams will find the most utility in private hosted repositories. Their low cost and basic authorization features are friendly to low-budget projects or private projects with minimal security requirements. Large companies or projects that need a higher degree of secrecy and have a suitable budget may find their needs better met by running their own private registry.

When you have a hard requirement on availability control, longevity control, or secrecy, then running a private registry may be your best option. In doing so, you gain control without sacrificing interoperability with Docker pull and push mechanisms or adding to the learning curve for your environment. People can interact with a private registry exactly as they would with a hosted registry.

• Regional image caches
• Team-specific image distribution for locality or visibility
• Environment or deployment stage-specific image pools
• Corporate processes for approving images
• Longevity control of external images

The biggest trade-off when going from hosted registries to private registries is gaining flexibility and control while requiring greater depth and breadth of engineering experience to build and maintain the solution. Docker image registries often consume large amounts of storage, so be sure to account for that in your analysis. The remainder of this section covers what you need in order to implement all but the most complicated registry deployment designs and highlights opportunities for customization in your environment.

docker run -d -p 5000:5000 \
        -v "$(pwd)"/data:/tmp/registry-dev \
        --restart=always --name local-registry registry:2

Companies that want tight version control on their external image dependencies will pull images from external sources such as Docker Hub and copy them into their own registry. You might do this to ensure that an important image does not change or disappear unexpectedly when the author updates or removes the source image. To get an idea of what it’s like working with your registry, consider a workflow for copying images from Docker Hub into your new registry:

docker image pull dockerinaction/ch9_registry_bound                  1

docker image ls -f "label=dia_excercise=ch9_registry_bound"          2

docker image tag dockerinaction/ch9_registry_bound \
    localhost:5000/dockerinaction/ch9_registry_bound

docker image push localhost:5000/dockerinaction/ch9_registry_bound   3

In running these four commands, you copy an example repository from Docker Hub into your local repository. If you execute these commands from the same location from which you started the registry, you’ll find that the newly created data subdirectory contains new registry data.

The tight integration you get with the Docker ecosystem can make it feel like you’re working with software that’s already installed on your computer. When internet latency has been eliminated, such as when you’re working with a local registry, it can feel even less like you’re working with distributed components. For that reason, the exercise of pushing data into a local repository isn’t very exciting on its own.

The next set of commands should impress on you that you’re working with a real registry. These commands will remove the example repositories from the local cache for your Docker daemon, demonstrate that they’re gone, and then reinstall them from your personal registry:

docker image rm \
    dockerinaction/ch9_registry_bound \
    localhost:5000/dockerinaction/ch9_registry_bound                  1

docker image ls -f "label=dia_excercise=ch9_registry_bound"

docker image pull localhost:5000/dockerinaction/ch9_registry_bound    2

docker image ls -f "label=dia_excercise=ch9_registry_bound"           3

docker container rm -vf local-registry                                4

• 1 Removes tagged reference
• 2 Pulls from registry again
• 3 Demonstrates that image is back
• 4 Cleans up local registry

This is the most flexible distribution method that involves Docker registries. If you need greater control over transport, storage, and artifact management, you should consider working directly with images in a manual distribution system.

Images are files, and you can distribute them as you would any other file. It’s common to see software available for download on websites, File Transport Protocol (FTP) servers, corporate storage networks, or via peer-to-peer networks. You could use any of these distribution channels for image distribution. You can even use email or a USB drive in cases where you know your image recipients. Manual image distribution methods provide the ultimate in flexibility, enabling varied use cases such as distributing images to many people at an event simultaneously or to a secure air-gapped network.

All the same criteria apply to manual distribution, but it’s difficult to discuss them without the context of a specific transportation method.

Building a fully functioning example will help you understand exactly what goes into a manual distribution infrastructure. This section will help you build an infrastructure with the File Transfer Protocol.

The example in this section uses two existing images. The first, dockerinaction/ch9 _ftpd, is a specialization of the centos:6 image; vsftpd (an FTP daemon) has been installed and configured for anonymous write access. The second image, dockerinaction/ch9_ftp_client, is a specialization of a popular minimal Alpine Linux image. An FTP client named LFTP has been installed and set as the entrypoint for the image.

To prepare for the experiment, pull a known image from Docker Hub that you want to distribute. In the example, the registry:2 image is used:

docker image pull registry:2

Once you have an image to distribute, you can begin. The first step is building your image distribution infrastructure. In this case, that means running an FTP server, which you will do on a dedicated network:

docker network create ch9_ftp
docker container run -d --name ftp-server --network=ch9_ftp -p 21:21 \
    dockerinaction/ch9_ftpd

This command starts an FTP server that accepts FTP connections on TCP port 21 (the default port). Don’t use this image in any production capacity. The server is configured to allow anonymous connections write access under the pub/incoming folder. Your distribution infrastructure will use that folder as an image distribution point.

Next, export an image to the file format. You can use the following command to do so:

docker image save -o ./registry.2.tar registry:2

Running this command exports the registry:2 image as a structured image file in your current directory. The file retains all the metadata and history associated with the image. At this point, you could inject all sorts of phases, such as checksum generation or file encryption. This infrastructure has no such requirements, and you should move along to distribution.

docker image load -i registry.2.tar

You can confirm that the image has been loaded from the archive by listing images for the registry repository again with docker image ls registry.

When you distribute image sources instead of images, you cut out all the Docker distribution workflow and rely solely on the Docker image builder. As with manual image publishing and distribution, source-distribution workflows should be evaluated against the selection criteria in the context of a particular implementation.

Image source distribution is one of the most common methods, despite having the most potential for complication. Popular version-control software handles many of the complications of source distribution’s expanded interface.

Supposing a producer has an existing project, Dockerfile, and GitHub repository, their distribution workflow will look like this:

git init
git config --global user.email "you@example.com"
git config --global user.name "Your Name"
git add Dockerfile
# git add *whatever other files you need for the image*
git commit -m "first commit"
git remote add origin https://github.com/<your username>/<your repo>.git
git push -u origin master

Meanwhile, a consumer would use a general command set that looks like this:

git clone https://github.com/<your username>/<your repo>.git
cd <your-repo>
docker image build -t <your username>/<your repo>. .

Image source distribution is divorced from all Docker distribution tools. By relying only on the image builder, you’re free to adopt any distribution toolset available. If you’re locked into a particular toolset for distribution or source control, this may be the only option that meets your criteria.

This chapter covers various software distribution mechanisms and the value contributed by Docker in each. A reader that has recently implemented a distribution channel, or is currently doing so, might take away additional insights into their solution. Others will learn more about available choices. In either case, it is important to make sure that you have gained the following insights before moving on:

• Having a spectrum of choices illustrates your range of options.
• You should use a consistent set of selection criteria in order to evaluate your distribution options and determine which method you should use.
• Hosted public repositories provide excellent project visibility, are free, and require little experience to adopt.
• Consumers will have a higher degree of trust in images generated by automated builds because a trusted third party builds them.
• Hosted private repositories are cost-effective for small teams and provide satisfactory access control.
• Running your own registry enables you to build infrastructure suitable for special use cases without abandoning the Docker distribution facilities.
• Distributing images as files can be accomplished with any file-sharing system.
• Image source distribution is flexible but only as complicated as you make it. Using popular source-distribution tools and patterns will keep things simple.