NGINX.COM
Web Server Load Balancing with NGINX Plus

We are very excited to announce the addition of custom dashboards to NGINX Amplify, our SaaS‑based tool for monitoring NGINX. Now you can create your own dashboards populated with highly customizable graphs of NGINX and system‑level metrics.

The 'My Custom Dashboard' screen in NGINX Amplify displays graphs of metrics specified by the user [How to monitor NGINX with the custom dashboard]

As you can see, not only can the metrics be summed (or averaged across several NGINX servers), but it is also possible to create additional “metric dimensions”; for example, the two graphs on the right in the screenshot each show the number of POST requests for a specific URI.

NGINX Amplify is currently in private beta. We want to thank everyone who provided their feedback and various suggestions. Everything you’ve told us so far has been incredibly helpful. If you haven’t yet, sign up for the private beta today.

Editor – NGINX Amplify is now in public beta.

To learn more about NGINX Amplify and see a demo, watch our on‑demand webinar, Improve App Performance & Reliability with NGINX Amplify. And check out all the posts about NGINX Amplify on our blog.

Now, let’s dig deeper into the custom dashboards.

Using NGINX Amplify’s Custom Dashboards to Monitor NGINX More Effectively

With this new feature, you can build bespoke sets of graphs for the metrics that NGINX Amplify collects.

Several use cases for a custom set of graphs that immediately come to mind are:

  • Checking NGINX performance for a particular application or microservice, e.g. based on the URI path
  • Displaying metrics per virtual server
  • Visualizing the performance of a group of NGINX servers – for example, frontend load balancers or an NGINX edge caching layer
  • Analyzing a detailed breakdown of HTTP status codes per application

To create a new custom dashboard, click CREATE DASHBOARD on the Dashboards drop‑down menu, as shown in the following screenshot.

To create a graph of custom metrics in NGINX Amplify, click 'CREATE DASHBOARD' on the 'Dashboards' drop-down menu [How to monitor NGINX with the NGINX custom dashboard

You can then add graphs and simple values to the dashboard. On the new dashboard screen, click Add Graph to bring up the following dialog box:

In the 'Add graph to dashboard' dialog in NGINX Amplify, you create a custom graph by defining the metric, source, type of aggregation, and filter to apply [How to monitor NGINX with the NGINX custom dashboard]

To add a graph, perform these steps:

  1. First pick one or more metrics of interest. You can combine multiple metrics on the same graph using the Add another metric button at the bottom.
  2. After the metric is selected, NGINX Amplify lists the objects for which it has already observed this particular metric. Select one or multiple objects here. In the example above the objects are the NGINX instances on the hosts called astra and otter.
  3. Select either sum or avg as the aggregation function.
  4. Last but not least, filtering is also available for NGINX metrics. If you click on Apply filter, you can then add multiple criteria in order to define specific “metric dimensions”. In the screenshot above, we are filtering by HTTP status code 201.
  5. Click Save when you’re done, and the graph is added to the dashboard. Obviously you can then edit the graph specs, move it around, resize, stack the graphs on top of each other, etc.

By default, we don’t really store all of the possible metric dimensions in the NGINX Amplify backend. A particular filter starts to slice the metric according to the specified dimensions only after the graph is created. Hence, it can be a while before the “filtered” metric is displayed on the graph – the end result depends on how quickly the log files are being populated with the new entries, but typically you will see the first data points in under 5 minutes.

How NGINX Amplify’s Custom Dashboards Help You Monitor NGINX

Because NGINX Amplify is not a log analyzer, we had to enhance the NGINX Amplify Agent in order to implement additional slicing for metric dimensions. We’ve added capabilities in the NGINX Amplify Agent that enable it not only to monitor NGINX in real time, but to parse NGINX access logs on the fly and extract all the necessary metrics without sending the raw log entries elsewhere. Moreover, the NGINX Amplify Agent can actually understand custom log formats automatically, and start looking for various newly defined metric dimensions.

In a sense, what we have now in the NGINX Amplify Agent is a combination of real‑time log analytics and collection of the standard metrics (that is, metrics from the stub_status module). Again, the NGINX Amplify Agent does only the real‑time log processing, and always on the same host where it is running.

Metric filters can be really powerful. By using the filters and creating additional metric dimensions, it becomes possible to build highly granular and very informative graphs. Bear in mind, though, that to enable the NGINX Amplify Agent to slice the metrics you must add the corresponding log variables to the active NGINX log format.

Technical Challenges

In addition to all of the above, we also wanted to describe the technical challenges that we had to overcome when implementing custom dashboards.

First of all, we have to deal with the many kinds of NGINX builds out there. Many people install NGINX from packages they find with their OS distribution, from our native repository at nginx.org, or maybe even from a third‑party repository. Overall, there are many versions and many different flavors of NGINX builds in the wild. Automatically identifying a particular running binary and finding all its metrics is sometimes a very tricky thing to do. If you don’t see the NGINX Amplify Agent successfully detecting your perfectly valid NGINX instances, give us a buzz (through the Intercom icon at the bottom right of the NGINX Amplify screen) and we’ll investigate it.

For NGINX metric collection per se we had to standardize on pretty much just two common denominators – stub_status and the log files. We also collect some additional metrics using psutil(). If you don’t see a particular metric being collected, be sure to check the docs, and feel free to ask for help as well.

Designing the Add graph to dashboard dialog box has been one of the hardest parts of implementing custom dashboards to help you better monitor NGINX. We really tried to make the dialog box as simple as possible, but given the complexity of the underlying metric collection mechanisms, your mileage may vary. Please feel free to send us any kind of observations and suggestions in regards to the usability of the graph configuration mechanism.

Last but not least, the NGINX Amplify Agent and the backend code had to be seriously revamped to support new metric breakdowns and additional metric dimensions. Sometimes bugs slip past our internal QA, so bear with us for a little while until it all settles.

Why Custom Dashboards?

When we first started to think about how the graphs in NGINX Amplify can best help you monitor NGINX, we had quite a few challenges and tradeoffs to manage. We ended up with an initial plan to implement scalable metric collection and just the core set of graphs first – in order to begin with something that would be useful for basic monitoring of NGINX performance.

As it turns out, a lot of our beta users found that easily seeing what their servers are doing, as well as identify and pinpoint various performance related problems, really helped them monitor NGINX more effectively.

However, as soon as we launched the private beta in November 2015, we also started to get more and more requests for somewhat more powerful graphing capabilities. In particular, a number of people immediately asked how they could build individual graphs for virtual hosts on their NGINX instances – and not just instance‑wide cumulative graphs.

Summary

There are many things that we’d like to improve in the dashboards going forward. We ponder a lot of additional details like graph design, better labeling, color themes, copying graphs between dashboards – and most certainly dashboard‑creation wizards. If you have your own ideas, send them over!

We hope you’ll like the new dashboards and will dig out even more useful information as you monitor NGINX on your servers!

As always, your feedback is highly valued, and very much appreciated. Sign up to try NGINX Amplify for yourself.

Editor – NGINX Amplify is now in public beta.

To learn more about NGINX Amplify and see a demo, watch our on‑demand webinar, Improve App Performance & Reliability with NGINX Amplify. And check out all the posts about NGINX Amplify on our blog.

Monitoring Highly Dynamic and Distributed Systems with NGINX Amplify

Cover image

Overcome challenges in collecting logs and metrics and acting on the results.



About The Author

NGINX part of F5

F5 NGINX

F5 is the company behind the popular open source project, NGINX. We offer a suite of technologies for developing and delivering modern applications. Together with F5, NGINX solutions bridge the gap between NetOps and DevOps, with multi‑cloud application services that span from code to customer.

About F5 NGINX

F5, Inc. is the company behind NGINX, the popular open source project. We offer a suite of technologies for developing and delivering modern applications. Together with F5, our combined solution bridges the gap between NetOps and DevOps, with multi-cloud application services that span from code to customer.

Learn more at nginx.com or join the conversation by following @nginx on Twitter.