NGINX.COM
Web Server Load Balancing with NGINX Plus

A New Home for the NGINX Blog on F5.com

NGINX has grown and changed significantly over the past few years. Since becoming a part of F5, we’ve been hard at work evolving our offerings to better serve the needs of the diverse global community of NGINX users. As a part of this evolution, we’re re-imagining how we deliver content to our valued users and contributors.

We’re excited to announce that starting on May 15th, 2024, we will be migrating the NGINX blog from its current home on nginx.com to f5.com. This strategic realignment of our content will make it easier to find the information and resources you need.

A New Chapter for NGINX Content

After the migration, the most important technical enablement blogs, tutorials, customer stories, and thought leadership blogs on topics like application delivery and security that are currently on NGINX.com will migrate to f5.com, while nginx.org will become a technical-focused hub for all things related to our thriving NGINX community.

With this migration, we set out to make your interactions with NGINX content more intuitive and rewarding. Expect a refreshed design, better navigation, and other new features that can help you access the content you need quickly. While the experience on our blog evolves, we’re committed to ensuring that the unique technical depth and community spirit you’ve come to expect from NGINX content doesn’t change.

Our Commitment to Open Source

We know that these changes may raise questions, especially within our open source community. Let us be clear: our commitment to open source is unwavering. That’s why we’re elevating and centralizing community-focused content with a dedicated home on nginx.org. Expect more content on open source tools, technical guides, and community initiatives. Your voices and contributions remain key in shaping the future of NGINX.

Join Us on This Journey

We’d like to invite you to be a part of this transition. On May 15th please take the time to familiarize yourself with NGINX content on f5.com. update your bookmarks, and explore the new design. Be aware, the occasional glitch can happen with web migrations like this. If you spot anything, please let us know via X or LinkedIn. We’d also love to hear any suggestions you have for our content — your feedback is invaluable.

4 Key Considerations for Unifying App Delivery and Security

Organizations today are facing the challenges of modernizing their applications and infrastructure to meet customers’ evolving needs. The transformation from monolithic architectures to microservices coupled with the shift from on-prem data centers to hybrid, multi-cloud environments is complex and fraught with risks. This transition to a perimeterless environment can leave organizations that rely on yesterday’s cybersecurity solutions open to today’s increasingly sophisticated cyberthreats. Considering the high number of apps, vendors, and tools across different environments in a modern organization’s tech stack, it’s no surprise that consistently managing and securing these complex app environments can be difficult and expensive. To overcome the challenges associated with tool and vendor sprawl, keep the four following strategic considerations in mind.

1. Improving Security Posture Without Adding Complexity

Imagine a walled city with one large central gate, and a group of guards tasked with protecting important people in the city. In this scenario it makes sense to station the guards at the gate so they can stop any potential threats before they enter the city. Now imagine the walls of the city are gone — it makes more sense for the guards to stay close to the people they’re protecting. Organizations’ app environments have evolved in a similar way, and the perimeter that acted as the walls of the city is no more. That’s why security should be close to the application, wherever it may be. To reduce tool sprawl, organizations should consider a high-performance, platform-agnostic solution that combines security and app delivery tools.

To ensure that applications are protected from all angles, an integrated Web Application Firewall (WAF) and Denial-of-Service (DoS) defense are critical. A robust WAF protects vulnerable apps from exploits, while DoS defense ensures application availability and performance. These security measures should also extend to protecting APIs, including REST, gRPC, and GraphQL, with advanced signatures, bot detection, and seamless integration with CI/CD pipelines through the Kubernetes API.

2. Gaining Visibility into App Health and Performance

Without visibility into app health and performance, ensuring that all apps are up to date and working properly can quickly turn into a management headache. A lack of visibility can make it hard or even impossible to maintain business SLAs, with prolonged downtimes, performance degradation, and cumbersome troubleshooting. An effective solution should give organizations access to detailed real-time and historical metrics that are necessary for troubleshooting and maintaining performance. For optimal performance and user experience, the solution should deliver scalability and low compute overhead.

3. Delivering Apps Faster Through Self-Service and Automation

Digital transformation is happening quickly, and as a result apps need to be delivered quickly too. Organizations can deliver apps faster and with greater agility by leveraging automation with self-service. This makes it quicker and easier for developers to release new application features through their CI/CD pipelines without using an outdated request-based provisioning model — developers shouldn’t need to submit a ticket whenever establishing, configuring, and re-configuring app connectivity.

4. Reducing Complexity Through Centralized Management

Modern application environments typically involve a wide range of tools to support app delivery. While it might not seem feasible to have one app delivery tool that does everything, monitoring and managing all an organization’s tools in one place is achievable. A centralized management tool can provide a single pane of glass for gaining visibility and control over security, routing, and monitoring policies at scale. This makes configuration management across diverse environments simpler, more consistent, and less expensive.

Adopting a holistic approach that encompasses security, visibility, automation, and centralized management enables organizations to deliver and secure apps from code to customer, improving customer experiences by reducing complexity and risk, increasing uptime, and providing better insights at scale. By prioritizing these key considerations, organizations can confidently navigate the complexities of modern application delivery.

To learn more about how NGINX can help you deliver apps simply, securely, and at scale and chat with one of our experts, click here. For more technically-oriented readers, download the NGINX Cookbook, our step-by-step guide on how to do all things NGINX.

Tutorial: Configure NGINX Usage Reporting

NGINX Plus R31 introduces a simpler and more efficient way to report NGINX Plus usage. Native integration of usage reporting eliminates the need for a separate NGINX Agent, making the reporting process seamless and hassle-free for customers in F5’s Flex Consumption Program (FCP), and for non-FCP customers on a term subscription. With this functionality built directly into NGINX Plus, instances can communicate necessary usage information to the NGINX Instance Manager periodically, without any performance impact. This feature is available for virtual machines and NGINX Ingress Controllers, but this blog focuses on NGINX Plus instances — you can find a guide on reporting the NGINX Ingress Controller to the NGINX Instance Manager here.

By simplifying setup, offering customization options, and providing fallback methods, we aim to make the required usage reporting an efficient and adaptable process for customers to implement and maintain compliance with FCP. For customers not yet on R31 or not planning to upgrade, alternative methods to enable usage reporting are available, either by installing the NGINX Agent or configuring HTTP health checks.

Note: To report NGINX Plus usage, you must install NGINX Instance Manager on a dedicated host. Refer to the installation guide to learn about different methods of installation.

There are three possible ways you can choose from to report NGINX Plus instances to F5.

  • NGINX Plus R31 users — Configuring native NGINX usage reporting
  • R31 or pre-R31 users — Installing NGINX Agent
  • Pre-R31 users — Configuring HTTP health check for NGINX Plus

Let’s explore them one by one:

Configure Native NGINX Usage Reporting

Upon installation, NGINX Plus establishes an automated connection to the NGINX Instance Manager via the default DNS entry “nginx-mgmt.local”. By default, every 30 minutes, NGINX Plus communicates with the Instance Manager and sends the usage information. If you need to change the Instance Manager DNS entry or alter the default connection interval to suit your needs better, add the mgmt{} block to your NGINX configuration. Additionally, it comes with other default directives, such as the UUID file, mTLS, DNS resolver, usage interval and usage_report endpoint. For a full list of customizable directives, refer to the comprehensive mgmtmodule documentation. With the ngx_mgmt_module, you can easily customize the directives, offering greater flexibility and convenience.

The following sections explain how to configure this feature for your environment.

Configure NGINX Plus

  1. It is advisable to use mTLS to enhance the security of your system. You can achieve this by generating a certificate, key, and CA key certificate for both the NGINX Plus instance and NGINX Instance Manager. For more information on how to generate keys, click here.
  2. Specify the path to the client certificate and private key file inside the management block of the configure file nginx.conf.
    		
       mgmt {
              ssl_protocols TLSv1.2 TLSv1.3;
              ssl_ciphers DEFAULT;
              ssl_certificate          /home/ubuntu/agent.crt;
              ssl_certificate_key      /home/ubuntu/agent.key
           }
    

  3. For a secure and reliable authentication process, specify the path to the CA chain to validate the authenticity of the certificates.
    
       mgmt {
              ssl_trusted_certificate  /home/ubuntu/ca.pem;
              ssl_verify               on;
              ssl_verify_depth         2;
           }
    

  4. For custom DNS resolvers, specify the resolver address in the mgmt{}block using the resolver directive.
    
       mgmt {
              resolver 3.X.X.X;
           }
    

  5. To configure a custom FQDN for NGINX Instance Manager, specify the FQDN in the usage_report directive. Then, add an A record to your local DNS that associates the hostname with the NGINX Instance Manager IP address.
    
       mgmt {
              resolver 3.X.X.X;
              usage_report endpoint=nms.local interval=15m;
           }
    

Example NGINX Plus configuration:


  mgmt {
         usage_report endpoint=nms.local interval=30m;
         resolver 3.X.X.X;
         ssl_protocols TLSv1.2 TLSv1.3;
         ssl_ciphers DEFAULT;
         ssl_certificate          /home/ubuntu/agent.crt;
         ssl_certificate_key      /home/ubuntu/agent.key;

         ssl_trusted_certificate  /home/ubuntu/ca.pem;
         ssl_verify               on;
         ssl_verify_depth         2;
     }

Save the NGINX configuration and reload NGINX Plus.
sudo nginx -s reload

Configure NGINX Instance Manager

Add the SSL certificate in /etc/nginx/conf.d/nms-HTTP.conf inside the server block. Click here to learn how to configure an SSL certificate in the NGINX Management Suite server.


server {
    listen 443 ssl http2;
    root /var/www/nms;
    server_name _;
    ssl_protocols       TLSv1.1 TLSv1.2;
    ssl_ciphers         HIGH:!aNULL:!MD5;
    ssl_session_cache   shared:SSL:10m;
    ssl_session_timeout 10m;
    ssl_certificate         /etc/nms/certs/server.crt;
    ssl_certificate_key     /etc/nms/certs/server.key;
    ssl_client_certificate  /etc/nms/certs/ca.pem;
    ssl_verify_client on;
 }

To view NGINX Plus usage, log in to NGINX Management Suite on your browser and navigate to the NGINX Instance Manager module. Then, select the NGINX Plus tab located at the bottom left of the page.

NGINX Plus inventory instances over time

Watch this video tutorial with step-by-step instructions:

Error Log Messages

Here is a list of error messages to troubleshoot in NGINX Plus instance when you are unable to see usage data in the NGINX Instance Manager:

Here are some examples of the error messages:

  • Dynamic resolver failures: Failure to resolve DNS resolver.
    2024/04/02 04:02:10 [error] 574079#574079: recv() failed (111: Connection refused) while resolving, resolver: 3.17.128.165:53
    2024/04/02 04:02:35 [warn] 574079#574079: usage report: nginx-mgmt.locals could not be resolved (110: Operation timed out)

  • System resolver failures: Failure to resolve the NGINX Instance Manager hostname
    2024/04/02 13:20:44 [info] 103888#0: usage report: host not found in resolver "nginx-mgmt.local"

  • SSL connection failures: Not able to verify or expired SSL certificate.
    2024/04/02 13:13:50 [warn] 103877#0: usage report: peer SSL connection failed

  • Peer communication failures: Failure to reach NGINX Instance Manager.
    2024/04/02 13:15:25 [warn] 103877#0: usage report: peer connection failed (-1: Unknown error)
    2024/04/02 13:23:32 [warn] 103877#0: usage report: failed (server returned: 404 Not Found)
    2024/04/02 19:53:45 [warn] 4648#4648: usage report: connection timed out

Install NGINX Agent

If you haven't moved to R31 yet or don't intend to do so soon, you can still report your NGINX Plus instances to the NGINX Instance Manager. You can achieve this by installing the NGINX Agent in your environment. Once the installation is complete, you can establish a connection with the Instance Manager and start transmitting usage data. To install the NGINX Agent, follow the instructions provided in this link.

To view NGINX Plus usage, log in to NGINX Management Suite on your browser and navigate to the NGINX Instance Manager module. Then, select the NGINX Plus tab located at the bottom left of the page.

NGINX Plus inventory instances over time

Watch this video tutorial with step-by-step instructions:

Configure HTTP Health Check for NGINX Plus

If you are not planning to install the NGINX Agent or upgrade to R31, you can still report your NGINX Plus instances. This can be achieved by configuring an HTTP health check through the NGINX configuration file. However, you must manually update the NGINX Plus configuration file to utilize this option. This method can be time-consuming and cumbersome, particularly if your environment has multiple instances. Therefore, it is advisable to consider these factors before choosing this option.

Configure NGINX Plus

  1. Open the NGINX Plus configuration and insert the code within the http {} block:
    
       keyval_zone zone=uuid:32K state=/var/lib/nginx/state/instance_uuid.json;
       keyval 1 $nginx_uuid zone=uuid;
       upstream receiver {
        zone receiver 64k;
    
        # REQUIRED: Update NMS_FQDN with NGINX Management Suite IP Address or hostname.
        # If configuring with hostname, please ensure to uncomment the resolver
        # directive below and define a DNS server that can resolve the hostname.
        server NMS_FQDN:443;
    
        # OPTIONAL: Update DNS_UP with DNS server IP address that can resolve
        # the hostname defined above.
        #resolver DNS_IP;
       }  
    
    map CERT $repo_crt {
        # OPTIONAL: Location of client certificate
          default /home/ubuntu/agent.crt;
       }  
    
    map KEY $repo_key {
        # OPTIONAL: Location of client certificate private key
          default /home/ubuntu/agent.key; 
       }
    
    
     server {
        location @ngx_usage_https {
            # OPTIONAL: Configure scheme (http|https) here
            proxy_pass https://receiver;
    
            # REQUIRED: If using NGINX APP PROTECT (NAP) on this instance, set nap=active on the following line:
    proxy_set_header Nginx-Usage "Version=$nginx_version;Hostname=$hostname;uuid=$nginx_uuid;nap=active";	 
    
            health_check uri=/api/nginx-usage interval=1800s;       # DO NOT MODIFY
            proxy_ssl_certificate     $repo_crt;                    # DO NOT MODIFY
            proxy_ssl_certificate_key $repo_key;                    # DO NOT MODIFY
        }
    
        location @self {
            health_check uri=/_uuid interval=1d;
            proxy_pass http://self;
        }
    
        location = /_uuid {
            if ($nginx_uuid !~ .) {
                set $nginx_uuid $request_id;
            }
            return 204;
        }
    
        listen unix:/tmp/ngx_usage.sock;
      }
    
     upstream self {
        zone self 64k;
        server unix:/tmp/ngx_usage.sock;
     }
    

  2. Update the NMS FQDN with your Instance Manager hostname or IP address. If you are using private DNS, uncomment and update your DNS IP address.
  3. To ensure the security of HTTP traffic to the NGINX Instance Manager, it's essential to specify the locations of the SSL certificate and key in the map CERT and map KEY blocks.

Save the changes and reload NGINX.
sudo nginx -s reload

To view NGINX Plus usage, log in to NGINX Management Suite on your browser and navigate to the NGINX Instance Manager module. Then, select the NGINX Plus tab at the bottom left of the page.

NGINX Plus inventory instances over time

Watch this video tutorial with step-by-step instructions:

Conclusion

We are committed to meeting you where you are and helping you streamline and automate usage reporting with ease. That's why we're providing multiple options for reporting flexibility. The recently released NGINX Plus R31 update makes it even simpler and more efficient to report monthly usage data on NGINX Plus instances — without impacting performance, while ensuring compliance with F5's requirements.

A Containerized WAF to Secure Apps and APIs

Applications today have different performance, privacy, and security considerations. Many likely have distinct optimal routing needs. Most are hosted in different environments. Organizations today need a unified solution that can be applied for applications deployed nearly anywhere—in public clouds, private clouds, and on premises. NGINX App Protect WAF delivers that unified solution.

In addition to its robust web application firewall (WAF) capabilities, significant improvements have been made to the architecture of NGINX App Protect WAF. Prior releases of NGINX App Protect (through 4.x) have been delivered as packaged software and this will continue with the NGINX App Protect WAF 4.x release.

But a new version of NGINX App Protect WAF, version 5.0, is designed for public cloud, private cloud, and on-premises deployments in a containerized form factor, which reduces the attack surface. NGINX App Protect WAF 5.0 introduces innovations that enable businesses to protect modern apps and APIs in containerized environments with a smaller footprint. NGINX App Protect WAF 5.0 not only supports both NGINX OSS and NGINX Plus, but also separates the control and data planes, enhancing agile development methodologies. NGINX App Protect 5.0 supports standalone policy management container and brings the following value to customers:

  • Prevents impact of heavy lift policy modifications to affect data traffic or data plane compute resources
  • Accelerates deploy and reload durations, making it faster to operate and scale

Because it’s designed for cloud and hybrid deployments, the containerized form factor approach of NGINX App Protect WAF 5.0 is optimized for DevOps teams using agile pipelines and CI/CD frameworks to build and maintain their applications.

NGINX App Protect 5.0 can address the desire for a unified, consistent app and API security solution, as it can be leveraged in public clouds, private clouds, and on premises. NGINX App Protect 5.0 also simplifies updates and upgrades. With its new architecture, NGINX App Protect 5.0 requires only the compiler to be rebuilt with updated signatures. And NGINX App Protect 5.0 supports NGINX in any form factor, including virtual machines and containers.

To be clear, though, NGINX App Protect 5.0 is not replacing NGINX App Protect 4.x. Both will be offered to customers to address their specific deployment needs, with NGINX App Protect 4.x continuing to be updated and offered as a packaged product and NGINX App Protect 5.0 and later being provided as a container supporting Docker or Kubernetes.

So, what does this mean for you?

For one, migrating to NGINX Plus is no longer a requirement, and there’s no need to re-customize your security solutions, which can save precious time and decrease total cost of ownership (TCO). Using containers, the native form factor for integration with modern applications, less of an attack surface is exposed with NGINX App Protect 5.0, enhancing security. You will also have the ability to protect applications running on NGINX OSS proxies if you choose to do so. This approach also offers better operability allowing SecOps and DevOps teams to more effectively maintain overall security posture management and implementation.

Enabling agile development and reducing the need to support new operating systems and version increments has been sought after, as it helps accelerate the pace of development. This approach also simplifies consumption by utilizing a container-ready form factor instead of self-creating containerized images and deploying per every instance for every release and signature update.

NGINX App Protect is a modern WAF designed to sit close to your applications, providing additional protection beyond a perimeter WAF. It can be fully integrated into DevOps and CI/CD frameworks to:

  • Enable strong security controls to be integrated seamlessly
  • Outperform other WAFs for improved user experience
  • Reduce complexity and tool sprawl while delivering modern apps

Let NGINX App Protect v5.0 shield your applications so that you can focus on the lifecycle and innovation of your products and solutions.

If you’re ready to try out NGINX App Protect v5.x, get started with this 30-day free trial from MyF5 or contact F5 sales today for more information.

Insights from the 2024 NGINX Cookbook: 4 Solutions to Today’s Top Application Delivery Problems

The 2024 edition of the NGINX Cookbook is here, and it’s packed full of new solutions to today’s most common application delivery problems. Since its initial release in 2004, NGINX has evolved beyond its web serving roots to become a versatile tool for load balancing, reverse proxying, and serving as an API gateway, including integration with Kubernetes through NGINX Ingress Controller and enhanced security features. To support these expanded NGINX deployments, the new version of the NGINX Cookbook offers over a hundred practical recipes for installing, configuring, securing, scaling, and troubleshooting your NGINX instances – invaluable whether you’re running NGINX Open Source on a smaller project or NGINX Plus in an enterprise environment. Keep reading for a quick look at sections of the Cookbook reflecting advancements in security and software load balancing.

Streamlining Service Communication with gRPC

Problem:

You need efficient communication between services, specifically the ability to terminate, inspect, route, or load balance gRPC method calls.

Solution:

Utilize NGINX as a proxy to terminate, inspect, route, and load balance gRPC method calls. This setup leverages HTTP/2’s capabilities for efficient communication while facilitating high performance and reliability of service interactions through effective load distribution and resiliency features like retries and circuit breaking.

Automating NGINX Provisioning in the Cloud

Problem:

To streamline deployments, you need to automate the provisioning and configuration of NGINX servers in cloud environments.

Solution:

Utilize tools like AWS EC2 UserData and Amazon Machine Images (AMIs) for AWS, or their equivalents in other cloud services, to automate the provisioning and configuration of NGINX servers.

Implementing HTTP Basic Authentication with NGINX

Problem:

You need to secure your application or content using HTTP basic authentication.

Solution:

Encrypt passwords using openssl and configure NGINX with auth_basic and auth_basic_user_file directives to require authentication. Ensure security by deploying over HTTPS.

Configuring NGINX Plus as a SAML Service Provider

Problem:

You want to enhance security by integrating NGINX Plus with a SAML identity provider (IdP) to safeguard resources through authentication.

Solution:

Set up NGINX Plus with the njs module and key-value store for SAML SP integration. Then configure SAML settings in NGINX Plus, adjust scripts and files for SP and IdP specifics.

Download the Cookbook for Free

Whether you’re just getting started with NGINX or an experienced user, this updated guide provides practical solutions to challenges you’ll likely face when deploying and scaling modern distributed applications. Empower yourself with the latest NGINX best practices and strategies. Download the free ebook today.

F5 NGINX Ingress Controller with Prometheus-operator for Out-of-the-Box Metrics

NGINX Ingress Controller from F5 NGINX combined with the Prometheus operator ServiceMonitor CRD makes gathering metrics from NGINX Ingress Controller deployments much easier and much faster using Helm.

The NGINX Ingress Controller helm chart now supports the ability to immediately take advantage of your existing Prometheus and prometheus-operator infrastructure, allowing you to deploy NIC and have metrics out of the box leveraging Prometheus ServiceMonitor.

This article walks you through what ServiceMonitor is, how you can install it, and how you can use the NGINX Ingress Controller helm chart to define these specific settings.

Prometheus ServiceMonitor

The Prometheus ServiceMonitor custom resource definition (CRD) allows you to declaratively define how a dynamic set of services should be monitored. The services monitored are defined using Kubernetes label selectors. This allows an organization to introduce conventions governing how metrics are exposed. Following these conventions, new services are automatically discovered, and Prometheus begins gathering metrics without the need to reconfigure the system.

ServiceMonitor is part of the Prometheus Operator. These resources describe and manage monitoring targets to be scraped by Prometheus. The Prometheus resource connects to ServiceMonitor using a ServiceMonitor Selector field. Prometheus can easily identify what targets have been marked for scraping. This gives you more control and flexibility to leverage ServiceMonitor resources in your Kubernetes cluster to monitor solutions like NGINX Ingress Controller.

To make things easier and provide out-of-the-box metrics for NGINX Ingress Controller, we recently added the ability to use Prometheus ServiceMonitor to our helm chart. This makes it quite easy to enable metrics for Prometheus to begin scraping right after deploying NGINX Ingress Controller.

To use this feature, we need to add a second service specifically created for metrics collection that ServiceMonitor will “attach” to. This will tell the Prometheus operator what service it should monitor (using the labels in the metadata) so it knows what and where to scrape.

Example of what a service for NGINX Ingress Controller would look like if part of deployment or helm files:


apiVersion: v1
kind: Service
metadata:
  name: nginx-ingress-servicemonitor
  labels:
    app: nginx-ingress-servicemonitor
spec:
  ports:
  - name: prometheus
    protocol: TCP
    port: 9113
    targetPort: 9113
  selector:
    app: nginx-ingress

The above will be part of the deployment. The label, app: nginx-ingress-servicemonitor “connects” to the serviceMonitor for Prometheus metric scraping.

Below is a sample serviceMonitor that would link to the above service named nginx-ingress-servicemonitor:


apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: nginx-ingress-servicemonitor
  labels:
    app: nginx-ingress-servicemonitor
spec:
  selector:
    matchLabels:
      app: nginx-ingress-servicemonitor
  endpoints:
  - port: prometheus

It is necessary to create a Prometheus resource, which is configured to look for the serviceMonitor resources, allowing Prometheus to quickly and easily know what endpoints to scrape for metrics.

In our example below, this resource tells Prometheus what items to monitor under the spec. In the below, we are monitoring spec.serviceMonitorSelector.matchLabels:. We can see that Prometheus is looking for matchLabels with app.nginx-ingress-servicemonitor in any namespace. This matches the serviceMonitor resource that will be deployed by the NGINX Ingress Controller helm charts and Helm.


apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: prometheus
  labels:
    prometheus: prometheus
spec:
  replicas: 1
  serviceAccountName: prometheus
  serviceMonitorNamespaceSelector:  {}
  serviceMonitorSelector:
    matchLabels:
      app: nginx-ingress-servicemonitor
  resources:
    requests:
      memory: 500Mi

Here is a diagram the connects the different pieces:

ServiceMonitor diagram
Figure 1: service-monitor object relationship

Installing Prometheus, prometheus-operator and Grafana

We are going to use the prometheus-community/kube-prometheus-stack to install the full deployment. This will install prometheus, prometheus-operator and Grafana. We are also going to specify we want to install this in the monitoring namespace for isolation.

Here is how we can install with helm:

helm install metrics01 prometheus-community/kube-prometheus-stack -n monitoring --create-namespace

Create and install Prometheus resource

Once Prometheus and the Prometheus CRDs are installed into the cluster, we can create our Prometheus resource. By deploying this ahead of time, we can “pre-plumb” our Prometheus setup with the labels we will use in the helm chart. With this approach, we can automatically have Prometheus start to look for NGINX Ingress Controller and scrape for metrics.

Our Prometheus resource will be deployed prior to installing NGINX Ingress Controller. This will allow the prometheus-operator to automatically pick up and scrape our NGINX Ingress controller after deployment, providing metrics quickly.


apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: prometheus
  namespace: default
  labels:
    prometheus: monitoring
spec:
  replicas: 1
  serviceAccountName: prometheus
  serviceMonitorNamespaceSelector:  {}
  serviceMonitorSelector:
    matchLabels:
      app: nginx-ingress-servicemonitor
  resources:
    requests:
      memory: 500Mi

Our example above is a basic example. The key part is the spec.serviceMonitorSelector.matchLabels value we specified. This value is what we are going to use when we deploy NGINX Ingress controller with the helm chart.

We want to provide Prometheus metrics out of the box. To do so, we are going to use NGINX Ingress Controller helm chart.

NGINX Ingress Controller helm values.yaml changes

We can review the values.yaml file for the helm chart. There is a Prometheus section we want to focus on as it has the required pieces needed to enable Prometheus, create the required service, and create a serviceMonitor resource.

Under the Prometheus section, we should see several settings:

prometheus.service
prometheus.serviceMonitor

We are going to enable both of the above settings, to generate the required service and serviceMonitor when using the helm chart.

Here is the specific section where we enable the service, enable serviceMonitor, and define the labels in the serviceMonitor section:


`servicemonitor` support was recently added to the NGINX Ingress controller helm chart. 
prometheus:
  ## Expose NGINX or NGINX Plus metrics in the Prometheus format.
  create: true

  ## Configures the port to scrape the metrics.
  port: 9113

  secret: ""

  ## Configures the HTTP scheme used.
  scheme: http

  service:
    ## Requires prometheus.create=true
    create: true

  serviceMonitor:
    create: true
    labels: { app: nginx-ingress-servicemonitor } 

Breaking out the values from the above:


prometheus:
  ## Expose NGINX or NGINX Plus metrics in the Prometheus format.
  create: true

Tells Helm you want to enable the NIC Prometheus endpoint. You can additionally define a port, scheme, and secret if required.

By setting the value of prometheus.service.create to true, Helm will automatically create the NIC ServiceMonitor service.


service:
    ## Creates a ClusterIP Service to expose Prometheus metrics internally
    ## Requires prometheus.create=true
    create: true

Lastly, we need to create the serviceMonitor. Setting this to true and adding the correct labels will create and add the labels that match our Prometheus resource.


serviceMonitor:
    ## Creates a serviceMonitor to expose statistics on the kubernetes pods.
    create: true
    ## Kubernetes object labels to attach to the serviceMonitor object.
    labels: { app: nginx-ingress-servicemonitor } 

The label links back to the name of the service.labels: { app: nginx-ingress-servicemointor }

To Summarize. Enable Prometheus, this exposes the Prometheus exporter capability of NIC. Define a Service object, this is how the Prometheus ServiceMonitor discovers NIC Prometheus exporter endpoints. Define a serviceMonitor object. This tells Prometheus ServiceMonitor to monitor this thing.

Now we can install NGINX Ingress Controller with helm.

Once we have modified our values.yaml, we can then proceed to install NGINX Ingress controller.

helm install nic01 -n nginx-ingress --create-namespace -f values.yaml .

After we deploy NGINX Ingress Controller, we can open up the Prometheus dashboard and navigate to status menu. From there we can navigate to the targets and service discovery views. Once Prometheus locates our new ServiceMonitor resource, it will begin to scrape the endpoint and collect metrics which are immediately picked up in the Prometheus dashboard.

Prometheus service discovery
Figure 2: Prometheus service discovery
Prometheus target
Figure 3: Prometheus target
Prometheus NGINX Query
Figure 4: Prometheus NGINX Query

We can see that using native Kubernetes tools like helm and Prometheus, NGINX Ingress Controller can make collecting metrics at the start of deployment a lot easier, providing “out of the box metrics”.

Here are reference documents to installing prometheus-operator:

https://prometheus-operator.dev/
https://github.com/prometheus-operator/prometheus-operator

Announcing NGINX Gateway Fabric Release 1.2.0

We are thrilled to share the latest news on NGINX Gateway Fabric, which is our conformant implementation of the Kubernetes Gateway API. We recently updated it to version 1.2.0, with several exciting new features and improvements. This release focuses on enhancing the platform’s capabilities and ensuring it meets our users’ demands. We have included F5 NGINX Plus support and expanded our API surface to cover the most demanded use cases. We believe these enhancements will create a better experience for all our users and help them achieve their goals more efficiently.

NGINX Gateway Fabric’s design and architecture overview

Figure 1: NGINX Gateway Fabric’s design and architecture overview


NGINX Gateway Fabric 1.2.0 at a glance:

  • NGINX Plus Support – NGINX Gateway Fabric now supports NGINX Plus for the data plane, which offers improved availability, detailed metrics, and real-time observability dashboards.
  • BackendTLSPolicy – TLS verification allows NGINX Gateway Fabric to confirm the identity of the backend application, protecting against potential hijacking of the connection by malicious applications. Additionally, TLS encrypts traffic within the cluster, ensuring secure communication between the client and the backend application.
  • URLRewrite – NGINX Gateway Fabric now supports URL rewrites in Route objects. With this feature, you can easily modify the original request URL and redirect it to a more appropriate destination. That way, as your backend applications undergo API changes, you can keep the APIs you expose to your clients consistent.
  • Product Telemetry – With product telemetry now present in NGINX Gateway Fabric, we can help further improve operational efficiency of your infrastructure by learning about how you use the product in your environment. Also, we are planning to share these insights regularly with the community during our meetings.

We’ll take a deeper look at the new features below.

What’s New in NGINX Gateway Fabric 1.2.0?

NGINX Plus Support

NGINX Gateway Fabric version 1.2.0 has been released with support for NGINX Plus, providing users with many new benefits. With the new upgrade, users can now leverage the advanced features of NGINX Plus in their deployments including additional Prometheus metrics, dynamic upstream reloads, and the NGINX Plus dashboard.

This upgrade also allows you the option to get support directly from NGINX for your environment.

Additional Prometheus Metrics

While using NGINX Plus as your data plane, additional advanced metrics will be exported alongside the metrics you would normally get with NGINX Open Source. Some highlights include metrics around http requests, streams, connections, and many more. For the full list, you can check NGINX’s Prometheus exporter for a convenient list, but note that the exporter is not strictly required for NGINX Gateway Fabric.

With any installation of Prometheus or Prometheus compatible scraper, you can scrape these metrics into your observability stack and build dashboards and alerts using one consistent layer within your architecture. Prometheus metrics are automatically available in the NGINX Gateway Fabric through HTTP Port 9113. You can also change the default port by updating the Pod template.

If you are looking for a simple setup, you can visit our GitHub page for more information on how to deploy and configure Prometheus to start collecting. Alternatively, if you are just looking to view the metrics and skip the setup, you can use the NGINX Plus dashboard, explained in the next section.

After installing Prometheus in your cluster, you can access its dashboard by running port-forwarding in the background.

kubectl -n monitoring port-forward svc/prometheus-server 9090:80

Prometheus Graph with NGINX Gateway Fabric connections accepted

Figure 2: Prometheus Graph showing NGINX Gateway Fabric connections accepted

The above setup will work even if you are using the default NGINX Open Source as your data plane as well! However, you will not see any of the additional metrics that NGINX Plus provides. As the size and scope of your cluster grows, we recommend looking at how NGINX Plus metrics can help quickly resolve your capacity planning issues, incidents, and even backend application faults.

Dynamic Upstream Reloads

Dynamic upstream reloads, enabled by NGINX Gateway Fabric automatically when installed with NGINX Plus, allow NGINX Gateway Fabric to make updates to NGINX configurations without a NGINX reload.

Traditionally, when a NGINX reload occurs, the existing connections are handled by the old worker processes while the newly configured workers handle new ones. When all the old connections are complete, the old workers are stopped, and NGINX continues with only the newly configured workers. In this way, configuration changes are handled gracefully even in NGINX Open Source.

However, when NGINX is under high load, maintaining both old and new workers can create a resource overhead that may cause problems, especially if trying to run NGINX Gateway Fabric as lean as possible. The dynamic upstream reloads featured in NGINX Plus bypass this problem by providing an API endpoint for configuration changes that NGINX Gateway Fabric will use automatically if present, reducing the need for extra resource overhead to handle old and new workers during the reload process.

As you begin to make changes more often to NGINX Gateway Fabric, reloads will occur more frequently. If you are curious how often or when reloads occur in your current installation of NGF, you can look at the Prometheus metric nginx_gateway_fabric_nginx_reloads_total. For a full, deep dive into the problem, check out Nick Shadrin’s article here!

Here’s an example of the metric in an environment with two deployments of NGINX Gateway Fabric in the Prometheus dashboard:

Prometheus graph with the NGINX Gateway Fabric reloads total

Figure 3: Prometheus graph showing the NGINX Gateway Fabric reloads total

NGINX Plus Dashboard

As previously mentioned, if you are looking for a quick way to view NGINX Plus metrics without a Prometheus installation or observability stack, the NGINX Plus dashboard gives you real-time monitoring of performance metrics you can use to troubleshoot incidents and keep an eye on resource capacity.

The dashboard gives you different views for all metrics NGINX Plus provides right away and is easily accessible on an internal port. If you would like to take a quick look for yourself as to what the dashboard capabilities look like, check out our dashboard demo site at demo.nginx.com.

To access the NGINX Plus dashboard on your NGINX Gateway Fabric installation, you can forward connections to Port 8765 on your local machine via port forwarding:

kubectl port-forward -n nginx-gateway 8765:8765

Next, open your preferred browser and type http://localhost:8765/dashboard.html in the address bar.

NGINX Plus Dashboard

Figure 4: NGINX Plus Dashboard overview

BackendTLSPolicy

This release now comes with the much-awaited support for the BackendTLSPolicy. The BackendTLSPolicy introduces encrypted TLS communication between NGINX Gateway Fabric and the application, greatly enhancing the communication channel’s security. Here’s an example that shows how to apply the policy by specifying settings such as TLS ciphers and protocols when validating server certificates against a trusted certificate authority (CA).

The BackendTLSPolicy enables users to secure their traffic between NGF and their backends. You can also set the minimum TLS version and cipher suites. This protects against malicious applications hijacking the connection and encrypts the traffic within the cluster.

To configure backend TLS termination, first create a ConfigMap with the CA certification you want to use. For help with managing internal Kubernetes certificates, check out this guide.


kind: ConfigMap
apiVersion: v1
metadata:
  name: backend-cert
data:
  ca.crt: 
         < -----BEGIN CERTIFICATE-----
	   -----END CERTIFICATE-----
          >

Next, we create the BackendTLSPolicy, which targets our secure-app Service and refers to the ConfigMap created in the previous step:


apiVersion: gateway.networking.k8s.io/v1alpha2
kind: BackendTLSPolicy
metadata:
  name: backend-tls
spec:
  targetRef:
    group: ''
    kind: Service
    name: secure-app
    namespace: default
  tls:
    caCertRefs:
    - name: backend-cert
      group: ''
      kind: ConfigMap
    hostname: secure-app.example.com

URLRewrite

With a URLRewrite filter, you can modify the original URL of an incoming request and redirect it to a different URL with zero performance impact. This is particularly useful when your backend applications change their exposed API, but you want to maintain backwards compatibility for your existing clients. You can also use this feature to expose a consistent API URL to your clients while redirecting the requests to different applications with different API URLs, providing an “experience” API that combines the functionality of several different APIs for your clients’ convenience and performance.

To get started, let’s create a gateway for the NGINX gateway fabric. This will enable us to define HTTP listeners and configure the Port 80 for optimal performance.


apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: cafe
spec:
  gatewayClassName: nginx
  listeners:
  - name: http
    port: 80
    protocol: HTTP

Let’s create an HTTPRoute resource and configure request filters to rewrite any requests for /coffee to /beans. We can also provide a /latte endpoint that is rewritten to include the /latte prefix for the backend to handle (“/latte/126” becomes “/126”).


apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: coffee
spec:
  parentRefs:
  - name: cafe
    sectionName: http
  hostnames:
  - "cafe.example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /coffee
    filters:
    - type: URLRewrite
      urlRewrite:
        path:
          type: ReplaceFullPath
          replaceFullPath: /beans
    backendRefs:
    - name: coffee
      port: 80
  - matches:
    - path:
        type: PathPrefix
        value: /latte
    filters:
    - type: URLRewrite
      urlRewrite:
        path:
          type: ReplacePrefixMatch
          replacePrefixMatch: /
    backendRefs:
    - name: coffee
      port: 80

The HTTP rewrite feature helps ensure flexibility between the endpoints on the client side and how they are mapped with the backend. It also allows traffic redirection from one URL to another, which is particularly helpful when migrating content to a new website or API traffic.

Although NGINX Gateway Fabric supports path-based rewrites, it currently does not support path-based redirects. Let us know if this is a feature you need for your environment.

Product Telemetry

We have decided to include product telemetry as a mechanism to passively collect feedback as a part of the 1.2 release. This feature will collect a variety of metrics from your environment and send them to our data collection platform every 24 hours. No PII is collected, and you can see the full list of what is collected here.

We are committed to providing complete transparency around our telemetry functionality. While we will document every field we collect, and you can validate what we collect by our code, you always have the option to disable it completely. We are planning to regularly review interesting observations based on the statistics we collect with the community in our community meetings, so make sure to drop by!

Resources

For the complete changelog for NGINX Gateway Fabric 1.2.0, see the Release Notes. To try NGINX Gateway Fabric for Kubernetes with NGINX Plus, start your free 30-day trial today or contact us to discuss your use cases.

If you would like to get involved, see what is coming next, or see the source code for NGINX Gateway Fabric, check out our repository on GitHub!

We have bi-weekly community meetings on Mondays at 9AM Pacific/5PM GMT. The meeting link, updates, agenda, and notes are on the NGINX Gateway Fabric Meeting Calendar. Links are also always available from our GitHub readme.

Our Design Vision for NGINX One: The Ultimate Data Plane SaaS

A Deeper Dive into F5 NGINX One, and an Invitation to Participate in Early Access

A few weeks ago, we introduced NGINX One to our customers at AppWorld 2024. We also opened NGINX One Early Access, and a waiting list is now building. The solution is also being featured at AppWorld EMEA and AppWorld Asia Pacific. Events throughout both regions will continue through June.

So the timing seems appropriate, in the midst of all this in-person activity, to share a bit more of our thinking and planning for NGINX One with our blog readers and re-extend that early access invitation to our global NGINX community.

Taking NGINX to Greater Heights

At the heart of all NGINX products lies our remarkable data plane. Designed and coded originally by Igor Sysoev, the NGINX data plane has stood the test of time. It is remarkably self-contained and performant. The code base has remained small and compact, with few dependencies and rare security issues. Our challenge was to make the data plane the center of a broader, complete product offering encompassing everything we build — and make that data plane more extensible, accessible, affordable, and manageable.

We also wanted to make NGINX a more accessible option for our large base of F5 customers. These are global teams for enterprise-wide network operations and security, many of which are responsible for NGINX deployments and ensuring that application development and platform ops teams get what they need to build modern applications.

Core Principles: No Silos, Consumption-Based, One Management Plane, Global Presence

With all this in mind, when we started planning NGINX One, we laid out a handful of design conventions that we wanted to follow:

  • Non-opinionated and flexible — NGINX One will be easy to implement across the entire range of NGINX use cases (web server, reverse proxy, application delivery, Kubernetes/microservices, application security, CDN).
  • Simple API interface — NGINX One will be easy to connect to any existing developer toolchain, platform, or system via RESTful APIs.
  • A single management system — NGINX One will provide one console and one management plane to run and configure everything NGINX. The console will be delivered ”as-a-service” with zero installation required and easy extensibility to other systems, such as Prometheus.
  • Consumption-based — With NGINX One, users will pay only for what they consume, substantially reducing barriers to entry and lowering overall cost of ownership.
  • Scales quickly, easily, and affordably in any cloud environment — NGINX One will be cloud and environment agnostic, delivering data plane, app delivery, and security capabilities on any cloud, any PaaS or orchestration engine, and for function-based and serverless environments.
  • Simplified security — NGINX One will make securing your applications in any environment easier to implement and manage, utilizing NGINX App Protect capabilities such as OneWAF and DDoS protection.
  • Intelligence for optimizing configurations — NGINX One will leverage all of NGINX’s global intelligence to offer intelligent suggestions on configuring your data plane, reducing errors, and increasing application performance.
  • Extensibility — NGINX One will be easy to integrate with other platforms for networking, observability and security, and application delivery. NGINX One will simplify integration with F5 Big-IP and other products, making it easier for network operations and security operations teams to secure and manage their technology estate across our product families.

NGINX One Is Designed to Be the Ultimate Data Plane Multi-Tool

We wanted to deliver all this while leveraging our core asset — the NGINX data plane. In fact, foundational to our early thinking on NGINX One was an acknowledgment that we needed to return to our data plane roots and make that the center of our universe.

NGINX One takes the core NGINX data plane software you’re familiar with and enhances it with SaaS-based tools for observability, management, and security. Whether you’re working on small-scale deployments or large, complex systems, NGINX One integrates seamlessly. You can use it as a drop-in replacement for any existing NGINX product.

For those of you navigating hybrid and multicloud environments, NGINX One simplifies the process. Integrating into your existing systems, CI/CD workflows, and cloud services is straightforward. NGINX One can be deployed in minutes and is consumable via API, giving you the flexibility to scale as needed. This service includes all essential NGINX products: NGINX Plus, NGINX Open Source, NGINX Instance Manager, NGINX Ingress Controller, and NGINX Gateway Fabric. NGINX One itself is hosted across multiple clouds for resilience.

In a nutshell, NGINX One can unify all your NGINX products into a single management sphere. Most importantly, with NGINX One you pay only for what you use. There are no annual license charges or per-seat costs. For startups, a generous free tier will allow you to scale and grow without fear of getting whacked with “gotcha” pricing. You can provision precisely what you need when you need it. You can dial it up and down as needed and automate scaling to ensure your apps are always performant.

NGINX One + F5 Big-IP = One Management Plane and Global Presence

To make NGINX easier to manage as part of F5 products, NGINX One better integrates with F5 while leveraging F5’s global infrastructure. To start with, NGINX One will be deployed on the F5 Distributed Cloud, adjoining NGINX One users with many additional capabilities. They can easily network across clouds with our Multicloud Network fabric without enduring complex integrations. They can configure granular security policies for specific teams and applications at the global firewall layer with less toil and fewer tickets. NGINX One users will benefit from our global network of points-of-presence, bringing applications much closer to end-users without having to bring in an additional content delivery network layer.

F5 users can easily leverage NGINX One to discover all instances of NGINX running in their enterprise environments and instrument those instances for better observability. In addition, F5’s security portfolio shares a single WAF engine, commonly referred to as “OneWAF”. This allows organizations to migrate the same policies they use in BIG-IP Advanced WAF to NGINX App Protect and to keep those policies synchronized.

A View into the Future

As we continue to mature NGINX One, we will ensure greater availability and scalability of your applications and infrastructure. We will do this by keeping your apps online with built-in high-availability and granular traffic controls, and by addressing predictable and unpredictable changes through automation and extensibility. And when you discover issues and automatically apply supervised configuration changes to multiple instances simultaneously you dramatically reduce your operational costs.

You will be able to resolve problems before your customers notice any disruptions by leveraging detailed AI-driven insights into the health of your apps, APIs, and infrastructure.
Identifying trends and cycles with historical data will enable you to accurately assess upcoming requirements, make better decisions, and streamline troubleshooting.

You can secure and control your network, applications and APIs while ensuring that your DevOps teams can integrate seamlessly with their CI/CD systems and tooling. Security will be closer to your application code and APIs and will be delivered on the shift-left promise. Organizations implementing zero trust will be able to validate users from edge to cloud without introducing complexity or unnecessary overhead. Moreover, you’ll further enhance your security posture by immediately discovering and quickly mitigating NGINX instances impacted by common vulnerabilities and exposures (CVEs), ensuring uniform protection across your infrastructure.

NGINX One will also change the way that you consume our product. We are moving to a SaaS-delivered model that allows you to pay for a single product and deliver our services wherever your teams need them –in your datacenter, the public cloud, or F5 Distributed Cloud. In the future more capabilities will come to our data plane, such as Webassembly. We will introduce new use cases like AI gateway. We are making it frictionless and ridiculously easy for you to consume these services with a consumption-based tiered pricing.

There will even be a free tier for a small number of NGINX instances and first-time customers. With consumption pricing you have a risk-free entry with low upfront costs.

It will be easier for procurement teams, because NGINX One will be included in all F5’s buying programs, including our Flexible Consumption Program.

No longer will pricing be a barrier for development teams. With NGINX One they will get all the capabilities and management that they need to secure, deliver, and optimize every App and API everywhere.

When Can I Get NGINX One, and How Can I Prepare?

In light of our recent news, many NGINX customers have asked when they can purchase NGINX One and what can they do now to get ready.

We expect NGINX One to be commercially available later this year. However, as mentioned above, customers can raise their hands now to get early access, try it out, and share their feedback for us to incorporate into our planning. In the meantime, all commercially available NGINX products will be compatible with NGINX One, so there is no need to worry that near-term purchases will soon be obsolete. They won’t.

In preparation to harness all the benefits of NGINX One, customers should ensure they are using the latest releases of their NGINX instances and ensure they are running NGINX Instance Manager as prescribed in their license.

The Ingress Controller: Touchstone for Securing AI/ML Apps in Kubernetes

One of the key advantages of running artificial intelligence (AI) and machine learning (ML) workloads in Kubernetes is a having a central point of control for all incoming requests through the Ingress Controller. It is a versatile module that serves as a load balancer and API gateway, providing a solid foundation for securing AI/ML applications in a Kubernetes environment.

As a unified tool, the Ingress Controller is a convenient touchpoint for applying security and performance measures, monitoring activity, and mandating compliance. More specifically, securing AI/ML applications at the Ingress Controller in a Kubernetes environment offers several strategic advantages that we explore in this blog.

Diagram of Ingress Controller ecosystem

Centralized Security and Compliance Control

Because Ingress Controller acts as a gateway to your Kubernetes cluster, it allows MLOps and platform engineering teams to implement a centralized point for enforcing security policies. This reduces the complexity of configuring security settings on a per-pod or per-service basis. By centralizing security controls at the Ingress level, you simplify the compliance process and make it easier to manage and monitor compliance status.

Consolidated Authentication and Authorization

The Ingress Controller is also the logical location to implement and enforce authentication and authorization for access to all your AI/ML applications. By adding strong certificate authority management, the Ingress Controller is also the linchpin of building zero trust (ZT) architectures for Kubernetes. ZT is crucial for ensuring continuous security and compliance of sensitive AI applications running on highly valuable proprietary data.

Rate Limiting and Access Control

The Ingress Controller is an ideal place to enforce rate limiting, protecting your applications from abuse, like DDoS attacks or excessive API calls, which is crucial for public-facing AI/ML APIs. With the rise of novel AI threats like model theft and data leaking, enforcing rate limiting and access control becomes more important in protecting against brute force attacks. It also helps prevent adversaries from abusing business logic or jailbreaking guardrails to extract data and model training or weight information.

Web Application Firewall (WAF) Integration

Many Ingress Controllers support integration with WAFs, which are table stakes for protecting exposed applications and services. WAFs provide an additional layer of security against common web vulnerabilities and attacks like the OWASP 10. Even more crucial, when properly tuned, WAFs protect against more targeted attacks aimed at AI/ML applications. A key consideration for AI/ML apps, where latency and performance are crucial, is potential overhead introduced by a WAF. Also, to be effective for AI/ML apps, the WAF must be tightly integrated into the Ingress Controller for monitoring and observability dashboards and alerting structures. If the WAF and Ingress Controller can share a common data plane, this is ideal.

Conclusion: Including the Ingress Controller Early in Planning for AI/ML Architectures

Because the Ingress Controller occupies such an important place in Kubernetes application deployment for AI/ML apps, it is best to include its capabilities as part of architecting AI/ML applications. This can alleviate duplication of functionality and can lead to a better decision on an Ingress Controller that will scale and grow with your AI/ML application needs. For MLOps teams, the Ingress Controller becomes a central control point for many of their critical platform and ops capabilities, with security among the top priorities.

Get Started with NGINX

NGINX offers a comprehensive set of tools and building blocks to meet your needs and enhance security, scalability, and observability of your Kubernetes platform.

You can get started today by requesting a free 30-day trial of Connectivity Stack for Kubernetes.

Scale, Secure, and Monitor AI/ML Workloads in Kubernetes with Ingress Controllers

AI and machine learning (AI/ML) workloads are revolutionizing how businesses operate and innovate. Kubernetes, the de facto standard for container orchestration and management, is the platform of choice for powering scalable large language model (LLM) workloads and inference models across hybrid, multi-cloud environments.

In Kubernetes, Ingress controllers play a vital role in delivering and securing containerized applications. Deployed at the edge of a Kubernetes cluster, they serve as the central point of handling communications between users and applications.

In this blog, we explore how Ingress controllers and F5 NGINX Connectivity Stack for Kubernetes can help simplify and streamline model serving, experimentation, monitoring, and security for AI/ML workloads.

Deploying AI/ML Models in Production at Scale

When deploying AI/ML models at scale, out-of-the-box Kubernetes features and capabilities can help you:

  • Accelerate and simplify the AI/ML application release life cycle.
  • Enable AI/ML workload portability across different environments.
  • Improve compute resource utilization efficiency and economics.
  • Deliver scalability and achieve production readiness.
  • Optimize the environment to meet business SLAs.

At the same time, organizations might face challenges with serving, experimenting, monitoring, and securing AI/ML models in production at scale:

  • Increasing complexity and tool sprawl makes it difficult for organizations to configure, operate, manage, automate, and troubleshoot Kubernetes environments on-premises, in the cloud, and at the edge.
  • Poor user experiences because of connection timeouts and errors due to dynamic events, such as pod failures and restarts, auto-scaling, and extremely high request rates.
  • Performance degradation, downtime, and slower and harder troubleshooting in complex Kubernetes environments due to aggregated reporting and lack of granular, real-time, and historical metrics.
  • Significant risk of exposure to cybersecurity threats in hybrid, multi-cloud Kubernetes environments because traditional security models are not designed to protect loosely coupled distributed applications.

Enterprise-class Ingress controllers like F5 NGINX Ingress Controller can help address these challenges. By leveraging one tool that combines Ingress controller, load balancer, and API gateway capabilities, you can achieve better uptime, protection, and visibility at scale – no matter where you run Kubernetes. In addition, it reduces complexity and operational cost.

Diagram of NGINX Ingress Controller ecosystem

NGINX Ingress Controller can also be tightly integrated with an industry-leading Layer 7 app protection technology from F5 that helps mitigate OWASP Top 10 cyberthreats for LLM Applications and defends AI/ML workloads from DoS attacks.

Benefits of Ingress Controllers for AI/ML Workloads

Ingress controllers can simplify and streamline deploying and running AI/ML workloads in production through the following capabilities:

  • Model serving – Deliver apps non-disruptively with Kubernetes-native load balancing, auto-scaling, rate limiting, and dynamic reconfiguration features.
  • Model experimentation – Implement blue-green and canary deployments, and A/B testing to roll out new versions and upgrades without downtime.
  • Model monitoring – Collect, represent, and analyze model metrics to gain better insight into app health and performance.
  • Model security – Configure user identity, authentication, authorization, role-based access control, and encryption capabilities to protect apps from cybersecurity threats.

NGINX Connectivity Stack for Kubernetes includes NGINX Ingress Controller and F5 NGINX App Protect to provide fast, reliable, and secure communications between Kubernetes clusters running AI/ML applications and their users – on-premises and in the cloud. It helps simplify and streamline model serving, experimentation, monitoring, and security across any Kubernetes environment, enhancing capabilities of cloud provider and pre-packaged Kubernetes offerings with higher degree of protection, availability, and observability at scale.

Get Started with NGINX Connectivity Stack for Kubernetes

NGINX offers a comprehensive set of tools and building blocks to meet your needs and enhance security, scalability, and visibility of your Kubernetes platform.

You can get started today by requesting a free 30-day trial of Connectivity Stack for Kubernetes.