NGINX.COM
Web Server Load Balancing with NGINX Plus

At nginx.conf2015 Ruslan Ermilov, developer at NGINX, Inc., described the development and use of dynamic modules for NGINX and NGINX Plus

The following is adapted from a presentation given by Ruslan Ermilov of NGINX, Inc. at nginx.conf 2015, held in San Francisco in September. You can view a recording of the complete presentation on YouTube. To download the slides, right-click here.

Table of Contents

0:00 Introduction
2:39 Module Growth
3:22 Core Modules
4:30 Event Modules
4:58 HTTP, Stream, and Mail Modules
5:47 More Modules
7:11 Static Modules
8:05 Fears and Excuses
9:57 The Demand for Dynamic Loading
11:12 Benefits of Dynamic Loading
12:12 Working Prototype
12:45 Building Dynamic Modules
13:15 Building Dynamic Modules – Example
14:04 Building Third Party Modules Dynamically
14:22 Loading Modules
15:36 Gory Details
17:29 More Gory Details
19:44 For Module Developers
21:05 Who is Involved
22:50 Questions and Answers

0:00 Introduction

My name is Ruslan – emphasis on the second syllable – but I don’t mind if you call me Ruslan, emphasis on the first syllable. I’m a developer for NGINX.

I’ve been with the company since early 2011. Among other things, I have been writing and heading support for IPv6. When we started NGINX Plus, I worked closely on dynamic configuration of upstreams and health checks. I also helped review Valentin’s code for SPDY and HTTP/2.

In this talk, I’m going to describe what we are doing in the ongoing project that will add support for dynamically loadable modules into NGINX.

2:39 Module Growth

Graph showing the increase in the number of modules in the NGINX core from 30 to nearly 100 over releases 0.1.0 through 1.9.5

Since its public launch, NGINX has worked with the idea of modules. The first public version, NGINX 0.1.0, was released in October 2004 and already had more than 30 modules. This included 22 HTTP modules, 5 core modules, and 9 event modules. Today, after a decade of continuous development, we have nearly 100 modules.

3:22 Core Modules

There are 10 core modules that implement generic types and objects, memory management, hashing, configuration file  parsing, logging, OS functions

Let’s take a look under the hood. Core modules implement basic things such as generic types and objects such as numbers, streams, arrays, lists, hashes, buffers, queues, and trees.

Core modules also provide a memory management API. We have a pool allocator, shared memory with slab allocator, and locking primitives for them such as mutexes, spin locks, read/write locks – almost an entire operating system. We implement several checksum and hashing algorithms, configuration file parsing, and configuration management interfaces.

We have logging, including syslog support, timekeeping, asynchronous resolver, connection management. The core modules are also responsible for specific tasks such as file and socket I/O, process management, IPC, timers, CPU affinity, and thread pools for offloading blocking I/O operations.

4:30 Event Modules

There are 9 event modules that implement connection processing on a per-platform basis

The event modules are responsible for connection processing. This includes epoll support on Linux, kqueue on BSD‑like systems, /dev/poll on all the Solaris and some other UNIX derivatives, event ports on Solaris 10 and later, and archaic methods such as select and poll. NGINX automatically chooses the most efficient method available for the platform.

4:58 HTTP, Stream, and Mail Modules

There are 63 HTTP modules for web serving, reverse proxy, and load balancing; 9 stream modules for TCP and UDP, and 7 Mail modules

HTTP modules are what make NGINX a good web server. We added the Stream modules in the NGINX 1.9.0 release. They allow NGINX to work as a generic TCP proxy server.

Mail modules have existed for a long time. These enable NGINX to proxy IMAP, POP3, and SMTP traffic.

5:47 More Modules

Third-party developers provide more than 120 modules; NGINX Plus has 12 exclusive modules and extends many others

Over the years, the growing NGINX community has developed more than 10 dozen third‑party modules, and we know quite a few companies which even write their own modules for NGINX.

Our commercial product NGINX Plus also adds another dozen modules. Among these are modules for media streaming – HLS and HDS. Modules that implement session logging instead of just the simple access logging available in the open source version. We have extended monitoring capabilities. We have dynamic configuration for upstream servers and health checks for them. We also have additional load‑balancing features such as NTLM support and least time balancing, as well as session stickiness [Editor – also called session persistence].

Besides new modules, NGINX Plus also extends existing modules with features such as limiting the maximum number of connections to upstream servers and keeping upstream configurations up‑to‑date by tracking DNS changes.

7:11 Static Modules

Static modules must be bundled in to the NGINX binary at compile time

So, how does it all work together? The set of modules that constitute an NGINX binary is currently fixed at compile time by the configure script. Standard modules are added to or excluded from the list using the --with_module and --without_module options.

The ability to compile in external modules appeared in early versions of NGINX, but the demand for dynamic module loading was low at the time. The number of external modules was pretty moderate, so supporting dynamic loading wasn’t a high priority task.

8:05 Fears and Excuses

hough the NGINX engineers considered dynamic modules early on, there are complicated dependency issues to overcome

Nevertheless, dynamic module loading was considered even early on. dlopen() allows you to easily load dynamically the piece of code into a new binary. But while dlopen() is a major system interface long available in nearly all UNIX‑like operating systems, it opens up a can of worms, namely dependency hell.

One of the typical issues with dynamic loading is when two modules are linked against different versions of the same library. If this happens, it can result in random crashes and other hard‑to‑diagnose problems.

So, what NGINX offers now is a little‑known feature: the ability to upgrade the executable on the fly. You can find more information here. Using this feature, it’s possible to run a new NGINX binary built with the modules you need, without any interruption in service. You just tell NGINX to upgrade the binary, providing the new one, and it will switch to the new process while the old process handles all existing connections until they complete.

It works well when you have source code for NGINX and source code for all the modules that you need as well as the freedom of mixing them in the way you need.

9:57 The Demand for Dynamic Loading

A major reason to support dynamic modules is that you can't add a module to binary packages without recompiling, and that's not even possible for NGINX Plus

Unfortunately, this is not always the case. Linux vendors ship NGINX in binary packages. Usually they provide several packages, each with its own set of modules. For example, on Ubuntu 14.04, there are five base packages of NGINX. These are NGINX Core, NGINX Light, NGINX Full, NGINX Extras, and NGINX Naxsi. NGINX Plus also comes as several packages.

What if you’re not happy with the set of modules in a package? You can’t just add another module to the crew; you have to recompile NGINX. So sometimes the provided packages may not suit you.

As the number of third‑party modules grew, and with the launch of NGINX Plus in August 2013, it became obvious that we should make an effort to allow modules to be loaded dynamically without the need for recompiling NGINX.

11:12 Benefits of Dynamic Loading

Benefits from supporting dynamic modules include reducing the number of binary packages and enabling NGINX Plus customers to add custom modules

Dynamic loading has the potential of reducing the number of packages that vendors have to ship, because individual modules can be shipped as distinct packages instead of just being part of base packages.

If you look at NGINX Plus, it is partly closed‑source, so you can’t recompile it with a custom set of modules. If we support dynamic loading, we can allow modules that pass our integrity checks to be loaded along with NGINX Plus.

Finally, there is growing demand from companies interested in running NGINX Plus with their own modules or supplying binary modules for open source NGINX.

12:12 Working Prototype

Screen trace shows the order in which binary objects are loaded in a prototype of dynamic modules

When it came to time to implement dynamic loading, we had a working prototype written in two hours. It could load the empty_gif and geoip modules dynamically.

You can see the traces of this attempt. If you’re familiar with UNIX, in this case FreeBSD, it shows you how different binary objects are loaded and in what order.

12:45 Building Dynamic Modules

The syntax for building dynamic modules extends the static syntax

The syntax to build dynamic modules simply extends existing syntax for building static modules. You simply say that you want to build some modules dynamically, and the rest is handled by the configure script. It generates the necessary makefile rules to build NGINX and the requested modules.

13:15 Building Dynamic Modules – Example

creen trace showing the build of the dynamically loadable GeoIP module

For example, if you build the GeoIP module dynamically, the dependency on the GeoIP library shifts from the main NGINX binary to the module. The module can then be packaged and distributed separately.

This is also why we don’t put modules that have external dependencies into the base package. We would like to avoid unnecessary dependencies on other vendors, other repositories, and so on. And for some modules, it could create legal issues because of different licensing.

14:04 Building Third‑Party Modules Dynamically

Syntax for building a dynamically loadable third-party module

Building a third‑party module dynamically is not much different. There is only a slight change to this existing syntax. I will talk more about third‑party modules later.

14:22 Loading Modules

he new load_module directive in the NGINX configuration file names each module to be loaded dynamically

For the configuration file, we added a new directive, load_module. It tells NGINX to load a module at program startup.

Once NGINX is running, you can change the list of modules by simply editing the configuration file, adding or removing the necessary load_module directives.

When you’re happy with your changes, you can signal NGINX to reload its configuration just as you do today. NGINX will check the syntax validity and try to apply the new configuration. If this fails for some reason, NGINX will roll back changes and continue to work with the old configuration.

If some modules became unused in between the changes in configuration, they will be unloaded after applying the new configuration.

15:36 Gory Details

Some requirement and restrictions are that certain modules must load in a prescribed order, some core modules cannot be made dynamic, and some 'complex modules' actually include multiple modules

Some modules have interdependencies that should be taken into account. For example, the three standard index modules – random_index, index, and autoindex – have to be loaded in this particular order or they won’t be able to run correctly. The same holds true for the filter modules.

Not all modules can be made dynamic. There are some modules, the core modules among them, that are an integral part of NGINX, and these can’t be cut into loadable objects.

There are also some complex modules. For example, Mail is actually a set of several modules, but they’re written in such a way that makes it impossible to separate specific modules. So right now, Mail is a single loadable module that provides eight NGINX modules.

Some third‑party modules like RTMP and Lua are also complex modules.

17:29 More Gory Details

Other requirements are restrictions resulting from the fact that NGINX doesn't have a cleanly separated API

Another obstacle is that NGINX doesn’t yet have a cleanly separated API. In contrast to libraries, when you program for NGINX, you may have to muck with its internal structures, call its functions; and sometimes you would even like to make some static functions external.

NGINX also doesn’t provide some abstraction layer like some other programs that use external modules. So, when you program an external module, it’s like you’re extending NGINX with the new module.

Well, what’s the problem with this? NGINX has a lot of conditional compilation defines that may influence the size and contents of various structures. This has to be taken into account when building a custom module.

So when you already have a NGINX binary and just want to build another module for it, you have to use not only the same NGINX header file versions, but also the same set of compilation defines.

There are also some special cases like, what to do when a new configuration removes some filter module, but we have to rollback to a previous configuration due to some issue.

Internally, the filter modules are organized as chains, and in this case we would have to restore the filter chain to its previous state as well. This is one of the tasks that is still not solved.

19:44 For Module Developers

Kinds of changes that developers of third-party modules must make to conver to dynamic modules

So what does this all mean for module developers?

If you are not interested in dynamically loading your module, you don’t have to worry. Everything will continue to work as before and you are not required to modify the source code of your module.

If you do, however, you just need to make some simple changes to your module’s configuration file. This example shows what such a config may look like. You specify the type and name of your module, supply the list of header files, an optional pass to them, then the list of source files and libraries the module depends on, if any. You can find another example config file and more details in our wiki. Everything is like before, just in a slightly different format.

The new auto/module script will handle the rest. It will generate makefile rules to build the modules statically or dynamically as you have requested.

More complex modules may require more changes to configuration files and maybe multiple calls to the auto/module script.

21:05 Who is Involved

Maxim Dounin and Ruslan Ermilov are the developers for dynamic modules at NGINX, Inc.

There are two developers involved in the project; these are their names. I am one of them.

Dynamic modules are now available in NGINX 1.9.11. The next NGINX Plus release, NGINX Plus R9, will also have dynamic modules. You can read more about it in our announcement blog post and find out implementation details in our wiki documentation.

22:50 Questions and Answers

Q: You mentioned that dynamic loading will let you build plug‑ins without the source. Does that means there’ll be a published API with header files to build against soon?

A:That was on my last slide. It’s still an ongoing project, so we don’t have any ready for public code yet. We are still experimenting, but we’re hoping to have a library and an API published soon. And it’s going to be possible to compile modules outside of NGINX.

Hero image
Free O'Reilly eBook: The Complete NGINX Cookbook

Updated for 2024 – Your guide to everything NGINX



About The Author

Ruslan Ermilov

Lead Programmer

About F5 NGINX

F5, Inc. is the company behind NGINX, the popular open source project. We offer a suite of technologies for developing and delivering modern applications. Together with F5, our combined solution bridges the gap between NetOps and DevOps, with multi-cloud application services that span from code to customer.

Learn more at nginx.com or join the conversation by following @nginx on Twitter.