Web Server Load Balancing with NGINX Plus

In the first part of this two‑part series of blog posts about NGINX Plus and the Internet of Things (IoT), we showed how NGINX Plus – as a fully featured application delivery controller (ADC) with support for TCP and UDP applications – increases the availability and reliability of IoT applications. In this second post, we discuss two ways to use NGINX Plus to improve IoT security. The two posts cover the following use cases:

Offloading TLS Termination from MQTT Servers

In the example use cases in the first post, all of the MQTT traffic is plaintext and unencrypted. To improve IoT security, it is best practice to use TLS to encrypt the MQTT data passing between clients and upstream servers, whenever TLS encryption is supported by the IoT devices or IoT gateway. Encryption is an effective way to protect data in motion as it crosses public networks, but in production environments with millions of devices it can put an enormous load on the MQTT servers.

As shown in Figure 1, NGINX Plus can offload the CPU‑intensive workload associated with TLS encryption from your MQTT servers (commonly called SSL offloading). This separation of concerns allows for the load‑balancing tier and MQTT data‑processing tier to scale independently and requires only a simple modification to our MQTT test environment.

(As in the first post, we use the Mosquitto command line tool as the client and HiveMQ instances running inside Docker containers as the MQTT brokers. For installation instructions, see Creating the Test Environment in the first post.)

To improve IoT security with TLS encryption, NGINX Plus performs TLS termination (often called SSL offloading) for MQTT devices
Figure 1. NGINX Plus performs TLS termination for MQTT clients

After furnishing the NGINX Plus instance with a TLS certificate key‑pair, we can use directives from the Stream SSL module to enable TLS termination.

This server block is identical to the configuration for session persistence in the previous post, except that line 2 specifies the standard port number for secure MQTT traffic, 8883, and lines 6–11 are added to configure the server to terminate TLS connections from clients. Explaining how to obtain and install a certificate is beyond the scope of this post; a production IoT deployment generally uses its own public key infrastructure (PKI). We also won’t go over the TLS‑related directives. For more information about TLS termination, see the NGINX Plus Admin Guide.

With this configuration in place we can use the Mosquitto MQTT client to publish an encrypted message. Notice that we specify the secure MQTT port (8883) and a file containing the public key of the certificate authority that issued the server certificate on our NGINX instance (cafile.pem).

$ mosquitto_pub -d -h -t "topic/test" -m "test123" -i "thing001" -p 8883 --cafile cafile.pem
Client thing001 sending CONNECT
Client thing001 received CONNACK
Client thing001 sending PUBLISH (d0, q0, r0, m1, 'topic/test', ... (7 bytes))
Client thing001 sending DISCONNECT
$ tail --lines=1 /var/log/nginx/mqtt_access.log [23/Feb/2017:11:41:56 +0000] TCP 200 23 4 thing001

Using Client Certificates to Authenticate MQTT Clients

[Editor – The following use case is just one of many for the NGINX JavaScript module. For a complete list, see Use Cases for the NGINX JavaScript Module.

The post has been updated to use the refactored session (s) object for the Stream module, which was introduced in NGINX JavaScript 0.2.4.]

Despite the success and widespread adoption of MQTT for IoT use cases, the protocol itself has very limited provision for verifying the identity of clients. Authentication is supported through the use of a username and password field in the MQTT CONNECT packet but in reality this is difficult to manage.

Although not officially supported by the MQTT specification, X.509 client certificates are commonly used to authenticate clients and are especially useful when combined with TLS encryption so that mutual authentication can take place:

  • The client verifies the identity of the server
  • The server verifies the identity of the client

NGINX Plus can combine TLS termination with client certificate authentication so that MQTT clients must provide a certificate, and that the common name (CN) of the certificate matches the MQTT ClientId. By linking the ClientId to the CN of the X.509 certificate, the MQTT server can be sure that messages received are from a trusted and genuine device.

To improve IoT security by authenticating MQTT clients, NGINX Plus processes X.509 client certificates (which here are effectively TLS certificates) and private keys
Figure 2. Provision of X.509 certificates and private keys for mutual authentication

NGINX Plus Configuration for MQTT Client Authentication

For this use case, we extend both the NGINX Plus configuration from the previous section (to enable authentication of client certificates) and the NGINX JavaScript code from the previous post (to match the certificate CN with the ClientId). Adding the following config snippet to the server block enables authentication of client certificates.

The ssl_verify_client on directive tells NGINX that clients must present certificates. In this example, client certificates are issued by an intermediate certificate authority and so we use the ssl_verify_depth directive to tell NGINX that there are two levels of issuer certificates to verify. The ssl_client_certificate directive specifies the location on disk of the public certificates for the certificate authorities (CAs) that issue certificates to clients; NGINX uses public CA certificates as part of the client authentication process.

JavaScript Code for MQTT Client Authentication

Finally, we extend the NGINX JavaScript code (mqtt.js) we created for the session persistence use case discussed in the previous post. The additional code validates that the MQTT ClientId presented in the CONNECT packet has the same value as the CN in the certificate issued to that same client.

We add the parseCSKVpairs function to extract the CN value from the X.509 certificate. It is called by another function (the getClientId function), and so must appear above it in the file.

The variable declarations and getClientId function on lines 14–45 are the same as lines 1–32 of the mqtt.js file we created for the session persistence use case.

Lines 48–49 are concerned with matching the ClientId with the certificate CN. The CN itself is contained within the certificate’s subject distinguished name, the value of which is available in the $ssl_client_s_dn variable. NGINX JavaScript has access to all of the NGINX variables through the s.variables object. This variable contains numerous attributes as a comma‑separated list of key‑value pairs.

The final lines (53–64) are the same as lines 34–45 in the mqtt.js file for the session persistence use case.

Testing MQTT Client Authentication

With this configuration in place we can send an authenticated and encrypted message to our test environment using the Mosquitto client. We can examine the certificate key‑pair that has been issued to thing001 by running this openssl x509(1) command.

$ openssl x509 -subject -noout < thing0001.crt
subject= /C=GB/L=Cambridge/ CA/CN=thing001

We can now supply this certificate key‑pair to our test environment.

$ mosquitto_pub -d -h -t "topic/test" -m "test123" -i "thing001" -p 8883 --cafile cafile.pem --cert thing0001.crt --key thing0001.key
Client thing001 sending CONNECT
Client thing001 received CONNACK
Client thing001 sending PUBLISH (d0, q0, r0, m1, 'topic/test', ... (7 bytes))
Client thing001 sending DISCONNECT
$ tail --lines=1 /var/log/nginx/mqtt_access.log [24/Feb/2017:14:37:08 +0000] TCP 200 23 4 thing001

If a client trying to establish a connection provides a ClientId that is mismatched with our certificate, or fails to provide a certificate at all, NGINX Plus terminates the connection immediately, so that unauthenticated messages never reach the upstream MQTT servers. Thus NGINX Plus provides additional protection to the MQTT servers from malicious or erroneous clients.

$ mosquitto_pub -d -h -t "topic/test" -m "test123" -i "BADTHING" -p 8883 --cafile cafile.pem --cert thing0001.crt --key thing0001.key
Error: The connection was lost.
$ mosquitto_pub -d -h -t "topic/test" -m "test123" -i "NOCERT" -p 8883 --cafile cafile.pem
Client NOCERT sending CONNECT
Error: The connection was lost.
$ tail --lines=2 /var/log/nginx/mqtt_access.log [24/Feb/2017:14:37:16 +0000] TCP 500 0 0 - BADTHING [24/Feb/2017:14:42:16 +0000] TCP 500 0 0 - -


Using NGINX Plus to offload both the encryption and authentication workloads from the MQTT servers increases IoT security as well as the overall performance and traffic capacity of IoT deployments. We’d love to hear about the use cases that you come up with for NGINX Plus and NGINX JavaScript, IoT or otherwise – please tell us about them in the comments section below.

To try NGINX JavaScript with NGINX Plus, start your free 30-day trial today or contact us to talk about your IoT project.

Hero image
Are Your Applications Secure?

Learn how to protect your apps with NGINX and NGINX Plus

About The Author

Liam Crilly

Sr Director, Product Management

About F5 NGINX

F5, Inc. is the company behind NGINX, the popular open source project. We offer a suite of technologies for developing and delivering modern applications. Together with F5, our combined solution bridges the gap between NetOps and DevOps, with multi-cloud application services that span from code to customer.

Learn more at or join the conversation by following @nginx on Twitter.