Harnessing the Power and Convenience of JavaScript for Each Request
Editor – This post has been updated to use the refactored HTTP request object (r
), which was introduced in NGINX JavaScript 0.2.2.
The post is the first of a number of blogs we’re publishing about NGINX JavaScript, which was formerly called nginScript. This post discusses why NGINX developed its own implementation of JavaScript, and presents a sample use case. Check out the other posts to explore additional use cases for both HTTP and TCP/UDP:
- Validating Request Bodies in “Deploying NGINX Plus as an API Gateway, Part 2: Protecting Backend Services”
- Diagnostic Logging with the NGINX JavaScript Module
- Announcing NGINX Plus R22 – Logging the exact set of headers sent by the client whenever an error is encountered
- Announcing NGINX Plus R21 – Chaining subrequests in a single code sequence without using callbacks; sending a copy of request headers to a security information and event management (SIEM) system
- Just One POST: Enabling Declarative DNS with F5 and the NGINX JavaScript Module
- Over-the-Air Updates to IoT Devices with NGINX
- Announcing NGINX Plus R18 – Logging a hashed (masked) version of the client IP address instead of the real address
- Virtual Patching with the NGINX JavaScript Module
- Batching API Requests with NGINX Plus and the NGINX JavaScript Module
- Announcing NGINX Plus R15 – Issuing an HTTP request to two different backends simultaneously, then forwarding the first response and ignoring the second; adding data integrity to application cookies
- Scaling Out Change Detection with JavaScript Hashing in “Detecting Homepage Defacement With Active Health Checks”
- Extending TCP/UDP Load Balancing with nginScript in “TCP/UDP Load Balancing with NGINX: Overview, Tips, and Tricks”
- Using Client Certificates to Authenticate MQTT Clients in “NGINX Plus for the IoT: Encrypting and Authenticating MQTT Traffic”
- Load Balancing MQTT for Session Persistence with the NGINX JavaScript Module in “NGINX Plus for the IoT: Load Balancing MQTT”
- Data Masking for User Privacy with the NGINX JavaScript Module
- Advanced Logging with the NGINX JavaScript Module in “Scaling MySQL with TCP Load Balancing and Galera Cluster”
- Announcing NGINX Plus R11 – Searching for key patterns in a message in a MySQL protocol stream to identify the SQL operation
- Using the NGINX JavaScript Module to Progressively Transition Clients to a New Server
The NGINX JavaScript module became generally available as a stable module in NGINX Open Source 1.11.10 and NGINX Plus R12. [The module was formerly called nginScript.] We have been working steadily on NGINX JavaScript since its launch in September 2015, adding the features and language support included in the stable module.
NGINX JavaScript is a unique JavaScript implementation for NGINX and NGINX Plus, designed specifically for server‑side use cases and per‑request processing. It extends NGINX configuration syntax with JavaScript code in order to implement sophisticated configuration solutions.
The use cases are extensive, especially as the NGINX JavaScript module is available for both HTTP and TCP/UDP protocols. Sample use cases for NGINX JavaScript include:
- Generating custom log formats with values not available from regular NGINX variables
- Implementing new load‑balancing algorithms
- Parsing TCP/UDP protocols for application‑level sticky sessions
There are of course many more possibilities for NGINX JavaScript, and more still that have yet to be implemented. Although we are pleased to announce general availability of the NGINX JavaScript module and recommend it for production use, there is a roadmap of planned improvements that will enable yet more use cases, such as:
- Inspecting and modifying the body of HTTP requests and responses (already supported for TCP/UDP traffic)
- Making HTTP subrequests from NGINX JavaScript code [added in NGINX Plus R15]
- Writing authentication handlers for HTTP requests (already supported for TCP/UDP traffic)
- Reading and writing files
Before discussing NGINX JavaScript in more detail, let’s first address two common misconceptions.
NGINX JavaScript Is Not Lua
The NGINX community has created several programmatic extensions over the years. At the time of writing, Lua is the most popular of these; it’s available as a module for NGINX and a certified third‑party module for NGINX Plus. The Lua module and add‑on libraries provide deep integration with the NGINX core and a rich set of functionality, including a driver for Redis.
Lua is a powerful scripting language. It, however, remains fairly niche in terms of adoption and is not typically found in the “skillset toolbox” of the frontend developer or DevOps engineer.
NGINX JavaScript does not seek to replace Lua and it will be some time before NGINX JavaScript has a comparable level of functionality. The goal of NGINX JavaScript is to provide programmatic configuration solutions to the widest possible community by using a popular programming language.
NGINX JavaScript Is Not Node.js
NGINX JavaScript does not aim to turn NGINX or NGINX Plus into an application server. In simple terms, the use cases for NGINX JavaScript are akin to middleware, as the code execution happens between the client and the content. Technically speaking, while Node.js shares two things with the combination of NGINX JavaScript and NGINX or NGINX Plus – an event‑driven architecture and the JavaScript programming language – the similarities end there.
Node.js uses the Google V8 JavaScript engine, whereas NGINX JavaScript is a bespoke implementation of the ECMAScript standards, designed specifically for NGINX and NGINX Plus. Node.js has a persistent JavaScript virtual machine (VM) in memory and performs routine garbage collection for memory management, whereas NGINX JavaScript initializes a new JavaScript VM and the necessary memory for each request and frees the memory when the request is completed.
JavaScript as a Server-Side Language
As mentioned above, NGINX JavaScript is a bespoke implementation of the JavaScript language. All other existing JavaScript runtime engines are designed to be executed within a web browser. The nature of client‑side code execution is different from server‑side code execution in many ways – from the availability of system resources to the possible number of concurrent runtimes.
We decided to implement our own JavaScript runtime in order to meet the requirements of server‑side code execution and fit elegantly with NGINX’s request‑processing architecture. Our design principles for NGINX JavaScript are these:
-
Runtime environment lives and dies with the request
The NGINX JavaScript module uses single‑threaded bytecode execution, designed for quick initialization and disposal. The runtime environment is initialized per request. Startup is extremely quick, because there is no complex state or helpers to initialize. Memory is accumulated in pools during execution and released at completion by freeing the pools. This memory management scheme eliminates the need to track and free individual objects or to use a garbage collector.
-
Non‑blocking code execution
NGINX and NGINX Plus’ event‑driven model schedules the execution of individual NGINX JavaScript runtime environments. When an NGINX JavaScript rule performs a blocking operation (such as reading network data or issuing an external subrequest), NGINX and NGINX Plus transparently suspend execution of the associated NGINX JavaScript VM and reschedule it when the event completes. This means that you can write rules in a simple, linear fashion and NGINX and NGINX Plus schedule them without internal blocking.
-
Implement only the language support that we need
The specifications for JavaScript are defined by the ECMAScript standards. NGINX JavaScript follows ECMAScript 5.1 with some ECMAScript 6 for mathematical functions. Implementing our own JavaScript runtime gives us the freedom to prioritize language support for server‑side use cases and ignore what we don’t need. We maintain a list of the currently supported language elements.
-
Close integration with request‑processing phases
NGINX and NGINX Plus process requests in distinct phases. Configuration directives typically operate at a specific phase and native NGINX modules often take advantage of the ability to inspect or modify a request at a particular phase. NGINX JavaScript exposes some of the processing phases through configuration directives to give control over when the JavaScript code is executed. This integration with the configuration syntax promises the power and flexibility of native NGINX modules with the simplicity of JavaScript code.
The table below indicates which processing phases are accessible via NGINX JavaScript at the time of writing, and the configuration directives that provide it.
Processing Phase HTTP Module Stream Module Access
– Network connection access control❌ ✅ js_access
Pre-read
– Read/write body❌ ✅ js_preread
Filter
– Read/write body during proxy❌ ✅ js_filter
Content
– Send response to client✅ js_content
❌ Log / Variables
– Evaluated on demand✅ js_set
✅ js_set
Getting Started with NGINX JavaScript – A Real‑World Example
NGINX JavaScript is implemented as a module that you can compile into an NGINX Open Source binary or dynamically load into NGINX or NGINX Plus. Instructions for enabling NGINX JavaScript with NGINX and NGINX Plus appear at the end of this article.
In this example we use NGINX or NGINX Plus as a simple reverse proxy and use NGINX JavaScript to construct access log entries in a specialized format, that:
- Includes the request headers sent by the client
- Includes the response headers returned by the backend
- Uses key‑value pairs for efficient ingestion into and searching with log processing tools such as the ELK Stack (now called Elastic Stack), Graylog, and Splunk
The NGINX configuration for this example is extremely simple.
As you can see, NGINX JavaScript code does not sit inline with the configuration syntax. Instead we use the js_include
directive to specify the file that contains all of our JavaScript code. The js_set
directive defines a new NGINX variable, $access_log_with_headers
, and the JavaScript function that populates it. The log_format
directive defines a new format called kvpairs which writes each log line with the value of $access_log_with_headers
.
The server
block defines a simple HTTP reverse proxy that forwards all requests to https://www.example.com. The access_log
directive specifies that all requests will be logged with the kvpairs format.
Let’s now look at the JavaScript code that prepares the log line. We have two functions:
kvHeaders
– A support function that converts theheaders
object to a string of key‑value pairs. Support functions must be declared before the function that calls them.kvAccessLog
– The function referenced by thejs_set
directive on line 2 of header_logging.conf. It receives an object argument (r
) that represents the request. This built‑in object can be passed to all HTTP NGINX JavaScript functions.
As can be seen in the
kvAccessLog
function, its return value is what is passed to the js_set
configuration directive. Bear in mind that NGINX variables are evaluated on demand and this in turn means that the JavaScript function defined by js_set
is executed when the value of the variable is required. In this example, $access_log_with_headers
is used in the log_format
directive and so kvAccessLog()
is executed at log time. Variables used as part of map
or rewrite
directives trigger the corresponding JavaScript execution at an earlier processing phase.
We can see this NGINX JavaScript‑enhanced logging solution in action by passing a request through our reverse proxy and observing the resulting log file entry, which includes request headers with the in.
prefix and response headers with the out.
prefix.
$ curl http://127.0.0.1/
$ tail --lines=1 /var/log/nginx/access_headers.log
2017-03-14T14:36:53+00:00 client=127.0.0.1 method=GET uri=/ status=200 in.Host=127.0.0.1 in.User-Agent=curl/7.47.0 in.Accept=*/* out.Cache-Control=max-age=604800 out.Etag=x22359670651+identx22 out.Expires='Tue, 21 Mar 2017 14:36:53 GMT' out.Last-Modified='Fri, 09 Aug 2013 23:54:35 GMT' out.Vary=Accept-Encoding out.X-Cache=HIT
Much of the utility of NGINX JavaScript is a result of its access to NGINX internals. This example utilizes several properties of the request (r
) object. The Stream NGINX JavaScript module (for TCP and UDP applications) utilizes a single session object (s
) with its own set of properties. For other examples of NGINX JavaScript solutions for both HTTP and TCP/UDP, see the list of posts at the top of this blog.
We’d love to hear about the use cases that you come up with for NGINX JavaScript – please tell us about them in the comments section below.
Enabling NGINX JavaScript for NGINX and NGINX Plus
- Loading the NGINX JavaScript Module for NGINX Plus
- Loading the NGINX JavaScript Module for NGINX Open Source
- Compiling NGINX JavaScript as a Dynamic Module for NGINX Open Source
Loading the NGINX JavaScript Module for NGINX Plus
NGINX JavaScript is available as a free dynamic module for NGINX Plus subscribers. For loading instructions, see the NGINX Plus Admin Guide.
Loading the NGINX JavaScript Module for NGINX Open Source
If your system is configured to use the official prebuilt packages for NGINX Open Source and your installed version is 1.9.11 or later, then you can install NGINX JavaScript as a prebuilt package for your platform.
-
Install the prebuilt package.
-
For Ubuntu and Debian systems:
$ sudo apt-get install nginx-module-njs
-
For RedHat, CentOS, and Oracle Linux systems:
$ sudo yum install nginx-module-njs
-
-
Enable the module by including a
load_module
directive for it in the top‑level ("main") context of the nginx.conf configuration file (not in thehttp
orstream
context). This example loads the NGINX JavaScript modules for both HTTP and TCP/UDP traffic.load_module modules/ngx_http_js_module.so; load_module modules/ngx_stream_js_module.so;
-
Reload NGINX Plus to load the NGINX JavaScript modules into the running instance.
$ sudo nginx -s reload
Compiling NGINX JavaScript as a Dynamic Module for NGINX Open Source
If you prefer to compile an NGINX module from source:
- Follow these instructions to build either or both the HTTP and TCP/UDP NGINX JavaScript modules from the open source repository.
- Copy the module binaries (ngx_http_js_module.so, ngx_stream_js_module.so) to the modules subdirectory of the NGINX root (usually /etc/nginx/modules).
- Perform Steps 2 and 3 in Loading the NGINX JavaScript Module for NGINX Open Source.