[Editor – We published a follow‑up to this post in August, 2018, taking into account updates to the BIG‑IP product line, and to both NGINX Plus and the commodity hardware it runs on, in the two years since this post was published. In several comparisons, NGINX Plus outperformed BIG‑IP for lower cost by even greater margins than in these tests.
Get all the details in NGINX Plus vs. F5 BIG‑IP: 2018 Price‑Performance Comparison.]
Recently a number of our customers, including AppNexus and IgnitionOne, have replaced leading hardware application delivery controller (ADC) appliances with NGINX Plus, and have seen both large cost savings and significant performance gains. They are not alone – other massive web properties have found that software implementations of ADC functions (including historically hardware‑intensive capabilities like SSL/TLS cryptography) have proven to be more than fast enough for their workloads.
We published an NGINX Sizing Guide based on the results from a range of tests measuring the performance of NGINX Plus on bare‑metal servers. In this article, we compare those performance numbers with the published metrics for F5 hardware appliances in three sizes.
Based on our testing, NGINX Plus on commodity hardware meets and often exceeds the performance of F5 BIG‑IP appliances at up to 83% cost savings.
Editor – For more information about replacing hardware ADCs with NGINX Plus, see these resources:
- 5 Reasons to Switch from F5 BIG‑IP to NGINX Plus
- Migrating Layer 7 Logic from F5 iRules and Citrix Policies to NGINX and NGINX Plus
- NGINX Plus vs. Citrix NetScaler: A Price‑Performance Comparison
- Migrating Load Balancer Configuration from F5 BIG‑IP LTM to NGINX Plus
- Migrating Load Balancer Configuration from Citrix NetScaler to NGINX Plus
In this blog, we’ll compare three simple, unambiguous performance metrics:
- HTTP requests per second (RPS)
- SSL/TLS transactions per second (TPS)
- HTTP throughput
For more details on the metrics, see Performance Metrics.
Metric values for F5 appliances are from a published datasheet, and pricing from two sources (CDW and Carahsoft), while the NGINX Plus performance numbers are from our Sizing Guide. Hardware cost for NGINX Plus is based on the list prices of Dell PowerEdge servers with the same specs as the Intel hardware that achieved the indicated results in our tests.
Note: The costs listed in the tables were accurate at the time of publication, but are subject to change over time.
Let’s review the findings.
NGINX Plus vs. F5 BIG‑IP 2000S
The table compares F5’s entry‑level application delivery controller, the F5 BIG‑IP 2000S, with NGINX Plus running on two different bare‑metal servers:
- The Dell PowerEdge R230 with a 4‑core Intel® Xeon® E3‑1220 v5 3.0GHz CPU and Intel XL710 2×40 Gbe NIC
- The Dell PowerEdge R430 with an 8‑core Intel® Xeon® E5‑2630 v3 2.4GHz CPU and Intel XL710 2×40 Gbe NIC
|F5 BIG‑IP 2000S||NGINX Plus
|Total Cost (Year 1)||$20,154||$3,900
|Total Cost (Year 3)||$24,472||$8,900
|Total Cost (Year 5)||$28,790||$13,900
1 Using OpenSSL 1.0.2d
NGINX Plus does not impose any artificial caps on throughput, meaning you get to use the full capacity of the hardware you’ve purchased.
NGINX Plus vs. F5 BIG‑IP 5050S
The table compares a mid‑range BIG‑IP appliance, the F5 BIG‑IP 5050S, with NGINX Plus running on a similarly sized bare‑metal server, the Dell PowerEdge R630 with a dual 16‑core Intel® Xeon® E5‑2697A v4 2.6GHz CPU and an Intel XL710 2×40 Gbe NIC.
|F5 BIG‑IP 5050S||NGINX Plus (Dell R630)|
|Total Cost (Year 1)||$58,494||$11,500
|Total Cost (Year 3)||$75,492||$18,500
|Total Cost (Year 5)||$92,490||$25,500
1 Using OpenSSL 1.0.2d
NGINX Plus vs. F5 BIG‑IP 11050
The table compares a high‑end BIG‑IP appliance, the F5 BIG‑IP 11050, with two NGINX Plus instances, each running on a Dell PowerEdge R630 with a dual 18‑core Intel® Xeon® E5‑2699 v3 2.3GHz CPUs and an Intel XL710 2×40 Gbe NIC. A single NGINX Plus instance can process up to 1.2 million requests per second, so two instances have about the same capacity as the BIG‑IP 11050, which can process up to 2.5 million requests per second.
|F5 BIG‑IP 11050S||NGINX Plus (Dell R630)|
|Hardware||$149,995||$22,000 ($11,000 each)|
|24×7 Support||$25,499||$7,000 ($3,500 each)|
|Total Cost (Year 1)||$175,494||$29,000
|Total Cost (Year 3)||$226,492||$43,000
|Total Cost (Year 5)||$277,490||$57,000
1 Using OpenSSL 1.0.2d
Reporting Results for Modern SSL/TLS Requirements
In accordance with current SSL/TLS best practices, we measured NGINX Plus’ SSL/TLS transactions per second (TPS) using the ECDHE‑RSA‑AES256‑GCM‑SHA384 cipher suite, which uses Ephemeral Elliptic curve Diffie–Hellman key exchange (ECDHE), AES, and SHA384. We also used an RSA 2048‑bit key for valid comparison with the performance figures on the F5 datasheets.
This cipher provides Perfect Forward Secrecy (PFS), which ensures that encrypted traffic captured now can’t be decrypted at a later time, even if the private key is compromised. PFS is becoming a “must have” in the current security climate. For example, Apple is mandating that iOS9 apps communicate using PFS.
F5 does not reveal the cipher used in their datasheet performance tests, and previous F5 benchmarks have not used PFS, which adds a performance penalty. F5 implements the ECDHE cipher in software on most platforms.
Readers should bear in mind the challenge of comparing SSL performance when different ciphers offer a tradeoff between security and speed.
Our customers report that they see significant cost savings when switching from hardware appliances to equivalent NGINX Plus solutions. Our own performance measurements and pricing analysis support this – for the simple use cases we examined, we saw between 80% and 84% cost savings in Year 1.
What makes NGINX Plus so different? We don’t bundle hardware with software, and we don’t apply an artificial performance cap on our software. You are free to select the most cost‑effective hardware for your needs. We don’t force you to accept hardware that doesn’t meet your company’s internal standards, nor are you obliged to overprovision hardware now in anticipation of growth in traffic or application complexity that might arise in 2 to 3 years’ time.
Finally, for the cost of a pair of hardware load balancers, you can get an application‑wide NGINX Plus subscription. With App Pricing, you can use unlimited NGINX Plus instances to support your entire application – load balancing, caching, web and media serving, and more – across all your production, test, and development environments. With standard, trusted technology throughout your application‑delivery stack, supported by the experts in the NGINX support team, what effect would that have on your operational and application delivery costs?
The data used to create this cost comparison was gathered from multiple sources:
- All NGINX Plus testing was done using three servers with a single 36‑core CPU in each. The servers were configured in a standard client → proxy → server topology.
- To get metrics for different numbers of CPU cores, the number of CPU cores in use was varied.
- Hardware specifications and performance metrics for BIG‑IP appliances are from the F5 BIG‑IP datasheet (we did not test F5 hardware ourselves).
The following performance metrics are compared in this report:
- Requests per second (RPS) – Measures the ability to process HTTP requests. In our tests for NGINX Plus, clients send requests over keepalive connections. NGINX Plus processes each request and forwards it to the web server over another keepalive connection.
- SSL/TLS transactions per second (TPS) – Measures the ability to process new SSL/TLS connections. In our tests for NGINX Plus, clients send a series of HTTPS requests, each on a new connection. NGINX Plus parses the requests and forwards them to the web server over an established keepalive connection. The web server sends back a 0 byte response for each request.
- Throughput – Measures the throughput sustained when serving large files over HTTP.