NGINX.COM
Web Server Load Balancing with NGINX Plus

Overview

Every business is uniquely challenged by the pressures of digitization. EPAM helps its customers become competitive – and stay competitive with its best-in-class software engineering with digital strategy and experience design, business consulting and technology innovation services. Its expert team and organized service practices help assess your market and design winning experiences, whether as a standalone engagement or a full end-to-end transformation program.

Challenge

EPAM Systems Inc. offers consulting‑based engineering services across a range of vertical markets. Since 1993, the company has partnered with the world’s leading companies to imagine, design, and engineer customer experiences that accelerate business transformation. One aspect of that work is InfoNgen, a service EPAM offers to customers which will alert them to important documents related to their market. InfoNgen crawls the Web to search millions of documents every day, sending related alerts to users in fields ranging from insurance to pharmacological research.

InfoNgen, originally a startup, was acquired by EPAM in 2010. Its technological core was moved to a private data center run by EPAM. After the acquisition, InfoNgen, as a business unit inside a larger consulting operation, required flexibility, stability, and affordability from its technology stack. By 2012, the biggest mandate for this business unit was to cut operational costs and to start migrating to the public cloud – which was, by that point, maturing to become a viable alternative for a robust, public‑facing service.

Windows containers were non‑existent essentially. So we decided to build our own containers on Windows. It’s not a big deal; it wasn’t a big challenge for us since all the operating system primitives were already there for building your own containers. So we just did it.
– Kirill Lutarevich, Senior Delivery Manager at EPAM

While the application was hosted in EPAM’s data center, the company was using a Citrix NetScaler [now Citrix ADC] appliance to perform load balancing and reverse proxy functionality. Unfortunately, this prevented a move to the cloud, as the load balancer was a piece of hardware. Additionally, Kirill Lutarevich, senior delivery manager at EPAM, says the Citrix license was around&bnsp;$50,000 per year, a high price for something which included functionality that wasn’t even being used by the team.

Solution

At first, Lutarevich replaced Citrix ADC with the free open source version of NGINX. This reduced load balancer and reverse proxy infrastructure costs to almost zero, and enabled the move to cloud, thanks to NGINX’s status as a multi‑platform, software‑based solution. The team quickly upgraded to NGINX Plus, however, in order to avail themselves of NGINX support and services. The upgrade still left the team with a tremendous cost savings over Citrix ADC.

InfoNegen’s solution for moving its legacy datacenter applications into the cloud and coupling them with microservices is now based entirely on NGINX Plus as the core traffic routing and balancing system.

EPAM's microservice architecture uses NGINX Plus
NGINX Plus load balances to InfoNgen’s Windows container‑based microservice architecture

InfoNgen uses an active/active pair of NGINX Plus instances in front of its microservices application. InfoNgen has 60–70 microservices running in production. The microservices are 80–85% hosted in custom‑built Windows containers; the rest are Linux. EPAM is not using the native container support found in Windows Server 2016, which came along four years too late for them. “The majority of our systems are on Windows Server 2012. Windows Server 2012 doesn’t support the concept of containers. So we built our own containers, custom containers,” explains Lutarevich.

InfoNgen uses ZooKeeper for service discovery. With InfoNgen’s microservices architecture, the IP address and port number of services are constantly changing. All its services register with ZooKeeper, which maintains a registry. NGINX Plus then pulls information about services from ZooKeeper, and routes requests accordingly.

Lutarevich notes that since the transition the team has had no issues or downtime related to its usage of NGINX Plus. Acting as an SSL terminator on the outside of its internal network, a single instance of NGINX Plus is able to handle all the traffic routing, balancing, and health monitoring tasks required by the system. The availability of robust health monitoring capabilities in NGINX Plus was, Lutarevich says, another deciding factor when the team chose to upgrade from the free version to the commercial edition.

Results

Cost Savings

Since the initial transition in 2012 to NGINX, and the subsequent move to NGINX Plus, Lutarevich was able to satisfy one of the most important needs of his company, which has been cutting infrastructure costs.

Our budget is very limited; we have to do what we can. We are under constant pressure to lower our infrastructure costs. What was not acceptable to us with Citrix [ADC] was the licensing costs, which were exorbitant.
– Kirill Lutarevich, Senior Delivery Manager at EPAM

By replacing Citrix ADC appliances with NGINX Plus, Lutarevich was able to cut costs significantly. Citrix ADC cost $50,000/year, according to Lutarevich, and NGIN Plus was a fraction of that cost.

Rolling Updates

The two internally hosted instances of NGINX Plus are used to enable rolling upgrades to services, says Lutarevich. NGINX Plus is able to reroute traffic to non‑upgraded nodes while software upgrades take place on others. The result is that NGINX Plus has allowed InfoNgen to remain online 24/7, without any downtime for upgrades, since the team implemented rolling upgrades with NGINX Plus.

In addition to rolling upgrades, NGINX Plus has provided a stable environment for both legacy Windows applications and modern Linux ones. While InfoNgen would love to move off of Windows in order to save money on hosting, this is simply not possible because of the size and importance of the core .NET application.

“Internally, we’ve been using it as a load balancer for our microservices architecture, so basically it’s like a system of registration and discovery for our microservices, which we controlled through the NGINX API. When we need to update or replace certain microservices endpoints, we turn them off, then turn them back on again, doing rolling upgrades by using the API provided by NGINX Plus,” says Lutarevich.

Community

The results of the move to NGINX Plus at InfoNgen are almost unexciting at this point. That’s because the transition was completed almost four years ago, and since that time, no issues have arisen in the NGINX layer. Lutarevich says that not only is NGINX stable and highly available; it’s also easy to find administrators and operators who know how to use it.

“NGINX is used almost everywhere, especially on the front‑end side,” says Lutarevich. “The people who have some skills or experience using NGINX – there are lots of them on the market. I can go ask questions on forums, on Stack Overflow, and hire someone there as well. There are lots of candidates with NGINX experience, so that makes it very useful for us.”

Next Steps

With the move to microservices, it doesn’t matter what operating system the services are hosted on, as InfoNgen utilizes a system of service discovery based on ZooKeeper and NGINX Plus. InfoNgen is able to build new services on Linux while still running its legacy Windows applications, all in the same environment, and with the same middleware and infrastructure in place for both types of systems.

Because InfoNgen is still using Windows Server 2012 for its .NET‑based microservices, supporting containers is tricky. This means containers are not supported out of the box in this operating system, and typical orchestration systems cannot work either, because they do not support Windows Server 2012. Still, InfoNgen has rolled its own containers for this environment and gotten them working, despite the older Windows version. This is bolstered by NGINX’s ability to rewrite the headers of incoming requests, allowing them to be forwarded internally to any number of systems, while external users continue to access the same URL.

“We are under constant pressure to lower our infrastructure costs, so people are asking us, ‘Are we using our EC2 instances effectively?’ If not, then we have to find a way to put as many microservices as possible on there. The only way to do that is through some sort of container infrastructure, like Docker,” explains Lutarevich.

But, because Docker does not work in Windows Server 2012, NGINX Plus is called upon to bridge the gap between old‑school infrastructure and modern application architecture. Fortunately, NGINX Plus is just as happy in a virtual machine as it is in a container, and it can route traffic to both types of systems, regardless of their underlying operating system or environment.

And, because NGINX is so versatile, it’s not affected by this custom container environment. For enterprises, customization is the most important feature for many software packages, and NGINX offers more customizability than any other load balancer or webserver on the market.

About EPAM

EPAM helps customers around the world become competitive – and stay competitive. They combine best-in-class software engineering with digital strategy and experience design, business consulting and technology innovation services. EPAM was founded in 1993 and is headquartered in Newtown, Pennsylvania.

Challenges

Cut operational costs and migrate to the public cloud.


Overview

EPAM Systems is a product development and digital platform engineering services company that works with its customers to deliver innovative solutions to solve complex business challenges.


Headquarters
Newtown, Pennsylvania
Founded in 1993
Use Cases
Solutions