NGINX.COM
Web Server Load Balancing with NGINX Plus

This blog post is the second in a series about how to monitor NGINX and make the most out of your web delivery infrastructure with NGINX Amplify:

  1. Setting Up NGINX Amplify in 10 Minutes
  2. Improving Server Configuration with NGINX Amplify Reports (this post)
  3. Using NGINX Amplify Custom Dashboards and Filters for Better NGINX Monitoring
  4. Inside NGINX Amplify: Insights from Our Engineers

NGINX Amplify, our new cloud‑based monitoring and assistance platform, has been very successful so far. Thousands of NGINX users are learning more about their installations, monitoring and improving server performance, and tuning configuration, security, and scalability problems using NGINX Amplify.

NGINX Amplify analyzes your NGINX configuration, and provides useful recommendations based on years of our collective experience with tuning, optimizing, and operating NGINX‑powered sites. We created this service so you can take advantage of this knowledge and have all your configuration files checked by our automated system. This post provides some deeper insight into one specific feature of NGINX Amplify – configuration reports – and how to use the service to optimize your NGINX configuration.

Architectural Components of NGINX Amplify

NGINX Amplify interacts with NGINX servers and users through the following components:

  • NGINX Amplify Agent – A lightweight software client that you install alongside your NGINX and NGINX Plus instances
  • NGINX Amplify Receiver – Located in the cloud and managed by NGINX
  • NGINX Amplify Web – Located in the cloud and managed by NGINX

The NGINX Amplify Agent is open source Python‑based software. The agent collects metadata and metrics from the operating system and all metrics available from NGINX.

NGINX Amplify processes and analyzes the configuration files for all your NGINX instances.

Every time the agent is started, or when the NGINX configuration is changed, the NGINX Amplify Agent parses configuration files. It does this for every running NGINX instance on the system. Parsed configuration data is sent to NGINX Amplify Cloud for automated analysis.

Configuration details that are sent to the NGINX Amplify Cloud include:

  • NGINX version information, including the build details and module information
  • Parsed configuration files, including:

    • Configuration file names
    • Listening IP addresses and ports
    • Server names
    • Locations
    • Variables in log_format
    • Configuration directives and their parameters
  • SSL certificate details as seen via OpenSSL client

Sensitive and private data is never sent to the cloud, for example:

  • SSL certificate files
  • SSL private keys
  • Password files with HTTP Basic authentication credentials or passwords for encrypted private keys
  • Raw logs

After analyzing the data from the agent, NGINX Amplify Web gives you valuable feedback about:

  • Current server and version information
  • Security vulnerabilities, including the CVE IDs of the vulnerabilities related to your NGINX version
  • HTTP, SSL/TLS, and HTTP/2 virtual server details
  • Static configuration analysis

Running the Configuration Report

To run the configuration report in NGINX Amplify, follow these steps:

  1. Sign into NGINX Amplify.
  2. Click the name of a system in the Systems list.
  3. Click the Reports tab. The NGINX configuration report for that system appears.
Screenshot of NGINX configuration analysis generated by the NGINX Amplify, which you can also use to monitor NGINX in real time
Figure 1. Example of an NGINX Amplify configuration report

The report consists of several sections. We will describe each section here.

Version

The Version section reports the version of NGINX software in use, the NGINX branch the version comes from (stable or mainline), the release date, and the most recent release on this branch.

Use this information to see if you are on the latest release from the branch you’ve chosen – for instance, if you are on the most recent mainline release. If not, consider upgrading to get the latest features and most recent bug fixes.

Overview

The Overview section includes information you need in order to review and update your current configuration:

  • Config path – The path of the main configuration file
  • Updated – The date the main configuration file was last modified
  • Config test errors – The number of times the NGINX Amplify Agent was unable to successfully run the nginx‑t command
  • Parser errors – The number of times the NGINX Amplify Agent was unable to fully parse a configuration file
  • Files – A count and list of configuration files that are in use by NGINX (see the documentation for the include directive for more details)
  • Third party modules – A list of the third‑party modules that NGINX was compiled with
  • Server – Number of current virtual servers
  • Server names – Number of virtual server names, in total and by type (exact, regex, or wildcard)
  • Locations – Number of server locations, in total and by type (exact, regex, or wildcard)
  • Upstreams – Number of upstream server groups
  • Listen IP/ports – A list of listening IP addresses and ports

Use this information to confirm the current status of your NGINX configuration files and to look for recent changes.

Security

The Security section does not appear if no security issues are found. This section lists security advisories against the currently installed version of NGINX. The information includes a description, a pointer to any relevant advisories, the severity level, and a list of versions of NGINX that are not vulnerable to this security advisory, so you can consider upgrading.

If you are not using an official build, there’s a potential complicating factor: some security fixes may be “backported” into older versions. For instance, you may see a report that says you are running version 1.8.1 with a vulnerability to CVE‑2016‑4450, which is shown as fixed for version 1.8.3. However, for non‑official builds, the fix may have already been applied to your system by your own security team or by a third party. Check the documentation for non‑official builds to see if security vulnerabilities listed in NGINX Amplify reports have been fixed for that build.

We recommend using the latest official builds whenever possible.

Virtual servers

The Virtual servers section provides the following information for the virtual servers defined for the NGINX instance being reported on:

  • Listen IP/port – The IP address and port on which the virtual server is listening
  • Server name – The name of the virtual server
  • Configuration file – The configuration file and line number where the virtual server is defined

SSL

Note: In order for NGINX Amplify Agent to check your SSL certificates, you must grant it filesystem access. See the instructions in our introductory NGINX Amplify blog post and the NGINX Amplify reference documentation for more information.

The SSL section tells you which version of SSL this NGINX instance is built with and which version it is dynamically linked with. This section lists the number of SSL‑enabled servers, HTTP/2‑enabled servers, and SSL certificates. Additional warnings are displayed when the SSL certificate expiration date is close or certificate renewal is overdue.

Your SSL certificates are analyzed in place, and private keys NEVER leave your servers.

Note: If you have performed an upgrade of the OpenSSL library, perform a full restart or upgrade of the NGINX service. A configuration reload (nginx‑sreload) does not change the running libraries.

Static analysis

The Static analysis section lists recommendations for improving your configuration file and settings. Recommendations can address security, completeness, error prevention, and other areas.

For each recommendation, NGINX Amplify lists it, along with filenames and line numbers where it was triggered. Recommendations include links to the reference documentation, where all directives and their parameters are fully described.

Examples of Static Analysis Messages

Figure 2 shows an example of an error and a warning resulting from static analysis of the configuration. Additional examples appear below the figure.

Here’s how to mitigate issues – and confirm that the mitigation has worked:

  1. Fix the configuration file(s) as suggested in the explanation of the error or warning.
  2. Reload NGINX.
  3. Wait about a minute for the agent to parse new configuration files and send data to NGINX Amplify Cloud.
  4. Refresh the report to verify that the error or warning no longer appears.
Sample NGINX Amplify reporting of warnings about NGINX configuration in the 'Static analysis' section of the 'Reports' tab
Figure 2. Example of warnings from a static analysis of NGINX configuration

Some additional examples of warnings and errors:

Warning: Can’t find /etc/nginx/conf.d/default.conf. This can potentially lead to a failed update when installing a new Plus nginx package. New package may bring a default.conf incompatible with your setup. It might get included then in the nginx configuration since nginx.conf contains include directive for conf.d/*.conf.

Warning: It is recommended to include default_server parameter for each unique listen IP:port combination. If none of the listen directives have the default_server parameter, then the first server with a particular IP:port tuple becomes the default one. Default server is the server that will process requests that can’t be matched against any of the configured server_name’s.

Warning: There should be normally a unique server_name directive per server.
Please ensure that a server_name contains all possible virtual server names for your site(s).

Warning: It is recommended that a proxy_pass always includes header definitions (at least Host header has to be explicitly defined). Consider adding the following additional directives to your nginx configuration:

proxy_set_header Host $host;

and optionally the following:

proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Protocol $scheme;

Error: Found parameter to a listen directive with a specific IP address, that puts it in conflict with a wildcard listen directive for the same port. While it is possible to make nginx listen on the same port with a wildcard socket and a specific IP address, such combination is not compatible with anything but default_server, ssl, spdy, proxy_protocol or http2 listen parameters.

Error: location has a regex pattern, but does not have a valid regex operator. Please make sure this is a desired behavior, since nginx location matching will not work with what is seemingly a regex pattern – but without proper regex operator.

If your configuration is clean and has no warnings or errors, the following message is displayed:

No issues found with the current configuration.

Controlling Configuration Reporting

You can control which kinds of configuration analysis the NGINX Amplify Agent performs. To change which reports are run for the covered systems, follow these steps:

  1. Click the Settings menu (the “hamburger” menu in the upper right corner of the NGINX Amplify Web page).
  2. Click Account. Account setting‑related options appear at the bottom of the page.
The 'Account Settings' panel controls which types of NGINX configuration analysis are included in NGINX Amplify reporting
Figure 3. Changing global settings for NGINX Amplify configuration reports

You can change three settings:

  • Enable NGINX config analyzer – If enabled, the NGINX Amplify Agent regularly reads the configuration files and sends configuration file data, including directives and parameters, to NGINX Amplify Cloud. This option is enabled by default.
  • Enable periodic “nginx ‑t” – If enabled, the NGINX Amplify Agent regularly runs the nginx‑t command to check whether the configuration has syntax errors. Typically a fully successful execution of the configuration test requires root privileges, and running it with the agent’s default permissions results in warnings in the NGINX error.log file. This option is disabled by default.
  • Analyze SSL certificates – If enabled, the NGINX Amplify Agent periodically reads SSL certificates to check their details and expiration time. This option is enabled by default.

You can further control which reporting options are allowed on a per‑system basis:

  • Open the Inventory menu (the “server rack” icon in the top navigation bar).
  • Select the NGINX instance you need to control and open the settings for it by clicking the icon circled in Figure 4.
  • Enable or disable the three options in the pop‑up window.
The 'NGINX settings' panel for an individual system controls which types of NGINX configuration analysis are included in NGINX Amplify reporting
Figure 4. Changing settings for NGINX Amplify configuration reports on an individual system

Summary

This post goes into depth about one capability of NGINX Amplify: configuration reporting. We’ve shown you how to get a report on your current configuration, customized by the type of information you want to review and the servers (Unix servers and NGINX servers) you wish to include.

Future blog posts will go into more depth about other ways to use NGINX Amplify to monitor NGINX and improve the performance of your site and apps. We look forward to your using NGINX Amplify, and to your feedback about how to improve it further.

Click the ? icon in the lower right corner of the NGINX Amplify Web page to provide feedback, or send us a message in the Comments below.

Sign up for free and start using NGINX Amplify today.

Monitoring Highly Dynamic and Distributed Systems with NGINX Amplify

Cover image

Overcome challenges in collecting logs and metrics and acting on the results.



About The Author

Nick Shadrin

Software Architect

About F5 NGINX

F5, Inc. is the company behind NGINX, the popular open source project. We offer a suite of technologies for developing and delivering modern applications. Together with F5, our combined solution bridges the gap between NetOps and DevOps, with multi-cloud application services that span from code to customer.

Learn more at nginx.com or join the conversation by following @nginx on Twitter.