NGINX Full Version

Docker Swarm Load Balancing with NGINX and NGINX Plus

At nginx.conf 2016 in Austin this September, I gave a presentation on using NGINX and NGINX Plus in a Docker Swarm cluster. In this post I discuss how to use NGINX and NGINX Plus for Docker Swarm load balancing in conjunction with the features introduced in Docker 1.12. All files I used during my demo at nginx.conf (and more) are available on GitHub for you to experiment with.


Docker version 1.12, released in late July 2016, integrates Docker Engine and Swarm and adds some new orchestration features, to create a platform similar to other container platforms such as Kubernetes. In Docker 1.12, Swarm Mode allows you to combine a set of Docker hosts into a swarm, providing a fault‑tolerant, self‑healing, decentralized architecture. The new platform also makes it easier to set up a Swarm cluster, secures all nodes with a key, and encrypts all communications between nodes with TLS.

At the same time, the Docker API has been expanded to be aware of services, which are sets of containers that use the same image (similar to services in Docker Compose, but with more features). You can create and scale services, do rolling updates, create health checks, and more. DNS service discovery and load balancing are built in, and you can also set up cluster‑wide overlay networks.

Topology for Docker Swarm Load Balancing

For this discussion and demo, I have three Swarm nodes – a manager and two workers. The manager node is where the Swarm commands are run. Swarm handles the scheduling, DNS service discovery, scaling, and container load balancing (represented in the figure by the small boxes) on all nodes.

Figure 1. A Docker Swarm cluster with a manager and two worker nodes

To provide private network communications between containers inside a cluster, containers can be connected to multiple internal overlay networks that span across all nodes in the cluster. Containers can be exposed outside of the cluster through the Swarm load balancer.

Figure 2. Internal and external network connectivity in a Docker Swarm cluster

The Docker Swarm load balancer runs on every node and can load balance requests across any of the containers on any of the hosts in the cluster. In a Swarm deployment without NGINX or NGINX Plus, the Swarm load balancer handles inbound client requests (represented by the green arrows in Figure 3) as well as internal service‑to‑service requests (represented by the red arrows).

Figure 3. Load balancing of client and service-to-service requests in a Swarm cluster without NGINX or NGINX Plus

Now that Swarm includes load balancing, why would you need another load balancer? One reason is that the Swarm load balancer is a basic Layer 4 (TCP) load balancer. Many applications require additional features, like these, to name just a few:

In addition, you might already have experience with a load balancer, and being able to use it with Swarm lets you to take advantage of the tooling and knowledge you are already using.

Using NGINX Open Source and NGINX Plus

NGINX Open Source and NGINX Plus are two load balancers that provide application‑critical features that are missing from the native Swarm load balancer.

Using NGINX Open Source

NGINX Open Source provides the features previously mentioned (SSL/TLS termination, etc.) and more, including:

The simplest way to use NGINX Open Source is to deploy it as a service, with one or more containers. The necessary ports for the NGINX service are exposed on the cluster, and the Swarm load balancers distribute requests on these ports to the NGINX containers.

Figure 4. Swarm load balancers distribute requests for the NGINX services across instances

For the purposes of this example, the service that NGINX provides is SSL/TLS termination. To illustrate how this works, we deploy a backend service A in the cluster and scale it to have three containers (two instances on one node and one instance on another, as shown in Figure 5). Swarm assigns a virtual IP address (VIP) to service A for use inside the cluster. We use this VIP in the NGINX configuration of the upstream group for service A, rather than listing the individual IP addresses of the containers. That way we can scale service A without having to change the NGINX configuration.

As shown in Figure 5, when a client makes a request for service A to the first Swarm node, the Swarm load balancer on that node routes the request to NGINX. NGINX processes the request, in this example doing SSL/TLS decryption, and routes it to the VIP for service A. The Swarm load balancer routes the (now unencrypted) request to one of the containers for service A, on any of the Swarm nodes.

Figure 5. NGINX provides SSL/TLS termination for external client requests

It is possible to have NGINX load balance requests directly to the backend containers, and also handle internal service‑to‑service requests, but only with a more complex solution that requires changing and reloading the NGINX configuration each time service A is scaled. I will discuss later how this is easily accomplished with NGINX Plus.

Using NGINX Plus

Some of the additional features provided by NGINX Plus are:

In the NGINX Open Source configuration described above, the Swarm load balancer both distributes external requests to the backend containers and handles service‑to‑service requests among them. The role of NGINX is SSL/TLS offloading.

When using NGINX Plus, client requests from external clients hit the Swarm load balancer first, but NGINX Plus does the actual load balancing to the backend containers (Figure 6). Having client requests hit the Swarm load balancer first provides an easy way of making NGINX Plus highly available.

Figure 6. The Docker Swarm load balancer forwards client requests to NGINX Plus for load balancing among service instances

Similarly, the Swarm load balancer receives interservice requests but NGINX Plus actually distributes them among the services (Figure 7).

Figure 7. The Docker Swarm load balancer forwards interservice requests to NGINX Plus for load balancing among service instances


To provide some examples of using Swarm for Docker load balancing with and without NGINX, I have created three demonstrations. All the files for these demonstrations are available on GitHub, along with detailed instructions. The three demonstrations are:

Docker Swarm Load Balancing

This demonstrates Docker Swarm load balancing of requests to a simple web app backend, without NGINX or NGINX Plus.

Docker Swarm Load Balancing with NGINX Open Source

This demo adds NGINX Open Source to provide SSL/TLS offload for external requests. The Swarm load balancer distributes requests to the same simple web app backend as in the previous demo, and handles internal service‑to‑service requests.

Docker Swarm Load Balancing with NGINX Plus

This demo has two parts. The first part uses NGINX Plus, which in addition to doing SSL/TLS offload, load balances requests directly to the backend containers and also handles internal service‑to‑service requests. It is integrated with Swarm service discovery, using dynamic DNS to frequently re‑resolve the domain name associated with the backends. NGINX Plus is load balancing two backend services, Service1 and Service2. It demonstrates internal service‑to‑service requests by having Service1 make a request to Service2.

Figure 8. NGINX Plus uses Swarm’s dynamic DNS service discovery mechanism when load balancing backend services

The second part shows how you can combine the NGINX Status API with the Docker Service API to automatically scale the backend containers. A Python program uses the NGINX Plus Status API to monitor the load on Service1 and Service2, and the Docker Swarm Service API to scale the backend containers up or down.

Figure 9. Docker Swarm uses NGINX Plus live activity monitoring to track service load for autoscaling purposes


The new features introduced in Docker 1.12 make Swarm a more powerful platform, but it can be enhanced by taking advantage of NGINX Open Source and even more by using NGINX Plus. The ability of NGINX Plus to dynamically reconfigure the backend containers to load balance using DNS, and the visibility provided by the Status API, make for a very powerful container solution.

To try NGINX Plus, start your free 30-day trial today or contact us to discuss your use cases.

And try out the demonstrations available at our GitHub repository.

Exit mobile version