Today we are launching the 2015 NGINX User Survey.
[Editor: As noted below, this survey closed on March 30, 2015.]
This survey gives us the opportunity to better understand your perspective on NGINX today and what might make us even more valuable for you in the future. The insights you share will be used to both help us plan our product roadmap, as well as influence how we communicate who we are and what we do.
The survey should take 15–20 minutes to complete and will remain open until March 30, 2015.
We’re excited to hear your thoughts and ideas. Thanks in advance for your input and for helping us continue to evolve NGINX!
In a previous blog post, we shared best practices for designing a microservices architecture, based on Adrian Cockcroft’s presentation at nginx.conf 2014 about his experience as Director of Web Engineering and then Cloud Architect at Netflix. In this follow‑up post, we’ll review his recommendations for retooling your development team and processes for a smooth transition to microservices.
Optimize for Speed, not Efficiency
Source: Xyzproject@Dreamstime.com
The top lesson that Cockcroft learned at Netflix is that speed wins in the marketplace. If you ask developers whether a slower development process is better, no one ever says yes. Nor do management or customers ever complain that your development cycle is too fast for them. The need for speed doesn’t just apply to tech companies, either: as software becomes increasingly ubiquitous on the Internet of Things – in cars, appliances, and sensors as well as mobile devices – companies that didn’t used to do software development at all now find that their success depends on being good at it.
Netflix made an early decision to optimize for speed. This refers specifically to tooling your software development process so that you can react quickly to what your customers want, or even better, can create innovative web experiences that attract customers. Speed means learning about your customers and giving them what they want at a faster pace than your competitors. By the time competitors are ready to challenge you in a specific way, you’ve moved on to the next set of improvements.
This approach turns the usual paradigm of optimizing for efficiency on its head. Efficiency generally means trying to control the overall flow of the development process to eliminate duplication of effort and avoid mistakes, with an eye to keeping costs down. The common result is that you end up focusing on savings instead of looking for opportunities that increase revenue.
In Cockcroft’s experience, if you say “I’m doing this because it’s more efficient”, the unintended result is that you’re slowing someone else down. This is not an encouragement to be wasteful, but you should optimize for speed first. Efficiency becomes secondary as you satisfy the constraint that you’re not slowing things down. The way you grow the business to be more efficient is to go faster.
Make Sure Your Assumptions are Still True
Many large companies that have enjoyed success in their market (we can call them incumbents) are finding themselves overtaken by nimbler, usually smaller, organizations (disruptors) that react much more quickly to changes in consumer behavior. Their large size isn’t necessarily the root of the problem – Netflix is no longer a small company, for example. As Cockcroft sees it, the main cause of difficulty for industry incumbents is that they’re operating under business assumptions that are no longer true. Or, as Will Rogers put it,
“It’s not what we don’t know that hurts. It’s what we know that ain’t so.”
Of course, you have to make assumptions as you formulate a business model, and then it makes sense to optimize your business practices around them. The danger comes from sticking with assumptions after they’re no longer true, which means you’re optimizing on the wrong thing. That’s when you become vulnerable to industry disruptors who are making the right assumptions and optimizations for the current business climate.
As examples, consider the following assumptions that hold sway at many incumbents. We’ll examine them further in the indicated sections and describe the approach Netflix adopted.
Computing power is expensive. This was true when increasing your computing capacity required capital expenditure on computer hardware. See Put Your Infrastructure in the Cloud.
Here are some ways to avoid holding onto assumptions that have passed their expiration date:
As obvious as it might seem, you need to make your assumptions explicit, then periodically review them to make sure they still hold true.
Keep aware of technological trends. As an example, the cost of solid‑state drive (SSD) storage continues to go down. It’s still more expensive than regular disks, but the cost difference is becoming small enough that many companies are deciding the superior performance is worth paying a bit more for.
[Editor – In this entertaining video, Fastly founder and CEO Artur Bergman explains why he believes SSDs are always the right choice.]
Talk to people who aren’t your customers. This is especially necessary for incumbents, who need to make sure that potential new customers are interested in their product. Otherwise, they don’t hear about the fact that they’re not being used. As an example, some vendors in the storage space are building hyper‑converged systems even as more and more companies are storing their data in the cloud and using open source storage management software. Netflix, for example, stores data on Amazon Web Services (AWS) servers with SSDs and manages it with Apache Cassandra. A single specialist in Java distributed systems is managing the entire configuration without any commercial storage tools or help from engineers specializing in storage, SAN, or backup.
Don’t base your future strategy on current IT spending, but instead on level of adoption by developers. Suppose that your company accounts for nearly all spending in the market for proprietary virtualization software, but then a competitor starts offering an open source‑based product at only 1% the cost of yours. If people start choosing it instead of your product, then at the point that your share of total spending is still 90%, your market share has declined to only 10%. If you’re only attending to your revenue, it seems like you’re still in good shape, but 10% of market share can collapse really quickly.
Put Your Infrastructure in the Cloud
Source: Fbmadeira@Dreamstime.com
We previously mentioned that in the past it was valid to base your business plan on the assumption that computing power was expensive, because it was: the only way to increase your computing capacity was to buy computer hardware. You could then make money by using this expensive resource in the right way to solve customer problems.
The advent of cloud computing has pretty much completely invalidated this assumption. It is now possible to buy the amount of capacity you need when you need it, and to pay for only the time you actually use it. The new assumption you need to make is that (virtual) machines are ephemeral. You can create and destroy them at the touch of a button or a call to an API, without any need to negotiate with other departments in your company.
One way to think of this change is that the self‑service cloud makes formerly impossible things instantaneous. All of Netflix’s engineers are in California, but they manage a worldwide infrastructure. The cloud enables them to experiment and determine whether (for example) adding servers in particular location improves performance. Suppose they notice problems with video delivery in Brazil. They can easily set up 100 cloud server instances in São Paulo within a couple hours. If after a week they determine that the difference in delivery speed and reliability isn’t large enough to justify the cost of the additional server instances, they can shut them down just as quickly and easily as they created them.
This kind of experiment would be so expensive with a traditional infrastructure that you would never attempt it. You would have to hire an agent in São Paulo to coordinate the project, find a data center, satisfy Brazilian government regulations, ship machines to Brazil, and so on. It would be six months before you could even run the test and find out that increased local capacity didn’t improve your delivery speed.
Create a High‑Freedom, High‑Responsibility Culture with Less Process
We previously observed that many companies create rules and processes to prevent problems. When someone makes a mistake, they add a rule to the HR manual that says “well, don’t do that again”. If you read some HR manuals from this perspective, you can extract a historical record of everything that went wrong at the company. When something goes wrong in the development process, the corresponding reaction is to add a new step to the procedure. The major problem with creating process to prevent problems is that over time you build up complex “scar tissue” processes that slow you down.
Netflix doesn’t have an HR manual. There is a single guideline: “Act in Netflix’s best interest”. The idea is that if an employee can’t figure out how to interpret the guideline in a given situation, he or she doesn’t have enough judgment to work there. If you don’t trust the judgment of the people on your team, you have to ask why you’re employing them. It’s true that you’ll have to fire people occasionally for violating the guideline. Overall, the high level of mutual trust among members of a team, and across the company as a whole, becomes a strong binding force.
The following books outline new ways of thinking about process if you’re looking to transform your organization:
The Goal: A Process of Ongoing Improvement by Eliyahu M. Goldratt and Jeff Cox. This book has become a standard management text at business schools since its original publication in 1984. Written as a novel about a manager who has only 90 days to improve performance at his factory or have it closed down, it embodies Goldratt’s Theory of Constraints in the context of process control and automation.
The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win by Gene Kim and Kevin Behr. As the title indicates, it’s also a novel, about an IT manager who has 90 days to save a project that’s late and over budget, or his entire department will be outsourced. He discovers DevOps as the solution to his problem.
Replace Silos with Microservices Teams
Most software development groups are separated into silos, with no overlap of personnel between them. The standard process for a software development project starts with the product manager meeting with the user‑experience and development groups to discuss ideas for new features. After the idea is implemented in code, the code is passed to the quality assurance (QA) and database administration teams and discussed in more meetings. Communication with the system, network, and SAN administrators is often via tickets. The whole process tends to be slow and loaded with overhead.
Source: Adrian Cockcroft
Some companies try to speed up by creating small “startup”‑style teams that handle the development process from end to end, or sometimes such teams are the result of acquisitions where the acquired company continues to run independently as a separate division. But if the small teams are still doing monolithic delivery, there are usually still handoffs between individuals or groups with responsibility for different functions. The process suffers from the same problems as monolithic delivery in larger companies – it’s simply not very efficient or agile.
Source: Adrian Cockcroft
Conway’s law says that the interface structure of a software system reflects the social structure of the organization that produced it. So if you want to switch to a microservices architecture, you need to organize your staff into product teams and use DevOps methodology. There are no longer distinct product managers, UX managers, development managers, and so on, managing downward in their silos. There is a manager for each product feature (implemented as a microservice), who supervises a team that handles all aspects of software development for the microservice, from conception through deployment. The platform team provides infrastructure support that the product teams access via APIs. At Netflix, the platform team was mostly AWS in Seattle, with some Netflix‑managed infrastructure layers built on top. But it doesn’t matter whether your cloud platform is in‑house or public; the important thing is that it’s API‑driven, self‑service, and automatable.
Source: Adrian Cockcroft
Adopt Continuous Delivery, Guided by the OODA Loop
A siloed team organization is usually paired with monolithic delivery model, in which an integrated, multifunction application is released as a unit (often version‑numbered) on a regular schedule. Most software development teams use this model initially because it is relatively simple and works well enough with a small number of developers (say, 50 or fewer). However, as the team grows it becomes a real issue when you discover a bug in one developer’s code during QA or production testing and the work of 99 other developers is blocked from release until the bug is fixed.
In 2009 Netflix adopted a continuous delivery model, which meshes perfectly with a microservices architecture. Each microservice represents a single product feature that can be updated independently of the other microservices and on its own schedule. Discovering a bug in a microservice has no effect on the release schedule of any other microservice. Continuous delivery relies on packaging microservices in standard containers. Netflix initially used Amazon machine images (AMIs) and it was possible to deploy an update into a test or production environment in about 10 minutes. With Docker, that time is reduced even further, to mere seconds in some cases.
Source: Adrian Cockcroft (http://www.slideshare.net/adrianco)
Observe refers to examining your current status to look for places where you can innovate. You want your company culture to implicitly authorize anyone who notices an opportunity to start a project to exploit it. For example, you might notice what the diagram calls a “customer pain point”: a lot of people abandoning the registration process on your website when they reach a certain step. You can undertake a project to investigate why and fix the problem.
Orient refers to analyzing metrics to understand the reasons for the phenomena you’ve observed at the Observe point. Often this involves analyzing large amounts of unstructured data, such as log files; this is often referred to as big data analysis. The answers you’re looking for are not already in your business intelligence database. You’re examining data that no one has previously looked at and asking questions that haven’t been asked before.
Decide refers to developing and executing a project plan. Company culture is a big factor at this point. As previously discussed, in a high‑freedom, high‑responsibility culture you don’t need to get management approval before starting to make changes. You share your plan, but you don’t have to ask for permission.
Act refers to testing your solution and putting it into production. You deploy a microservice that includes your incremental feature to a cloud environment, where it’s automatically put into an A/B test to compare it to the previous solution, side by side, for as long as it takes to collect the data that shows whether your approach is better. Cooperating microservices aren’t disrupted, and customers don’t see your changes unless they’re selected for the test. If your solution is better, you deploy it into production. It doesn’t have to be a big improvement, either. If the number of clients for your microservice is large enough, then even a fraction of a percent improvement (in response time, say) can be shown to be statistically valid, and the cumulative effect over time of many small changes can be significant.
Now you’re back at the Observe point. You don’t always have to perform all the steps or do them in strict order, either. The important characteristic of the process is that it enables you quickly to determine what your customers want and to create it for them. Cockcroft says “it’s hard not to win” if you’re basing your moves on enough data points and your competitors are making guesses that take months to be proven or disproven.
The state of art is to circle the loop every one to two weeks, but every microservice team can do it independently. With microservices you can go much faster because you’re not trying to get the entire company going around the loop in lockstep.
When it’s time to deliver your apps, NGINX Plus offers an application delivery platform that provides the superior performance, reliability, and scalability your users expect. Fully adopting a microservices‑based architecture is easier and more likely to succeed when you move to a single software tool for web serving, load balancing, and content caching. NGINX Plus combines those functions and more in one easy to deploy and manage package. Our approach empowers developers to define and control the flawless delivery of their microservices, while respecting the standards and best practices put into place by a platform team. Click here to learn more about how NGINX Plus can help your applications succeed.
Video Recordings
Fast Delivery
nginx.conf2014, October 2014
Migrating to Microservices, Part 1
Silicon Valley Microservices Meetup, August 2014
Migrating to Microservices, Part 2
Silicon Valley Microservices Meetup, August 2014
Elasticsearch is an advanced, high‑performance, and scalable open source search engine that provides full‑text search and real‑time analytics for structured and unstructured data. Its features are available using a RESTful API over HTTP, making it easy to fit into modern web architectures. Because NGINX Plus load balances HTTP traffic with outstanding performance at very high concurrency rates, it’s ideal for use as a reverse proxy load balancer as you scale up to more than one Elasticsearch server.
Editor – The first Elasticsearch User Conference – Elastic{ON} – took place at Pier 27 in San Francisco on March 9–11, 2015, shortly after the original publication of this blog. It was the single largest gathering of Elasticsearch, Logstash, and Kibana expertise anywhere in the world at the time. NGINX, Inc. was a conference sponsor.
The Advantages of Deploying NGINX Plus with Elasticsearch
Elasticsearch has a few features designed for scalability, but you can free up resources on your Elasticsearch servers by offloading the load balancing of requests to NGINX Open Source or NGINX Plus, which has even more enterprise‑grade features.
Putting NGINX or NGINX Plus in front of a single Elasticsearch server provides benefits, such as request logging, but the real value comes when you scale to multiple Elasticsearch servers.
In addition to logging each API request, NGINX and NGINX Plus:
Support a very large number of client connections (with or without keepalives enabled) with a much smaller of number of persistent connections (using keepalives) to the Elasticsearch servers.
Load balance requests across the Elasticsearch servers.
Cache data locally to reduce repeated requests to the Elasticsearch servers for the same content. NGINX Plus further provides an HTTP API for purging objects from the cache.
(NGINX Plus only) Provides application health checks to continually check whether the Elasticsearch servers are up and functioning. Down servers are automatically removed from the load balancing rotation and returned to it when they come back up.
(NGINX Plus only) Reports a wealth of real‑time metrics to improve monitoring and management. A demonstration of the metrics is available at demo.nginx.com.
(NGINX Plus only) Supports dynamic reconfiguration of upstream server groups via the NGINX Plus API. You can add and remove servers from the upstream group, mark them up or down, and change their weights.
The following table compares the functionality available with Elasticsearch by itself, combined with NGINX Open Source, and combined with NGINX Plus:
Elasticsearch client
NGINX Open Source
NGINX Plus
Horizontal scalability
✅
✅
✅
Keepalive optimization
❌
✅
✅
Centralized HTTP access logs
❌
✅
✅
Queuing and concurrency control
❌
❌
✅
Response caching
❌
✅
✅
Failover of failed connections
✅
✅
✅
Active monitoring of Elasticsearch nodes
✅ (some clients)
❌
✅
Advanced load‑balancing methods
❌
✅
✅
Weighted load balancing
❌
✅
✅
Dynamic reconfiguration
❌
❌
✅
Status monitoring
❌
❌
✅
General‑purpose HTTP load balancing
❌
✅
✅
When you use NGINX Plus in a high‑availability active‑passive configuration to load balance a cluster of Elasticsearch servers, another benefit is that you can configure the Elasticsearch clients to direct requests to NGINX Plus, rather than to the Elasticsearch servers. This means you can scale the Elasticsearch server cluster as needed without having to update the clients.
Deploying NGINX Plus with Elasticsearch
Deploying NGINX Plus with Elasticsearch is very straightforward. The directives in the following sample NGINX Plus configuration define settings for load balancing two Elasticsearch servers:
upstream – Load balance requests across two Elasticsearch servers with IP addresses 192.168.187.132 and 192.168.187.133 that listen on port 9200.
proxy_cache_valid – Cache valid responses for 10 minutes and 404 responses for 1 minute.
health_check – Actively check the health of the servers, taking them out of the load‑balancing rotation when they are down and adding them back into the rotation when are healthy again. The health check references the match block, which not only makes sure that the server returns a status code of 200, but checks that the response is formatted in JSON and that it returns the string expected from Elasticsearch:
"status" : 200
Final server block – Enable the NGINX Plus API to collect statistics and listen on port 8080 for requests for the dashboard.html page which displays the statistics.
# in the 'http' context
proxy_cache_path /var/cache/nginx/cache keys_zone=elasticsearch:10m inactive=60m;
upstream elasticsearch_servers {
zone elasticsearch_servers 64K;
server 192.168.187.132:9200;
server 192.168.187.133:9200;
}
match statusok {
status 200;
header Content-Type ~ "application/json";
body ~ '"status" : 200';
}
server {
listen 9200;
status_zone elasticsearch;
location / {
proxy_pass http://elasticsearch_servers;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_cache elasticsearch;
proxy_cache_valid 200 302 10m;
proxy_cache_valid 404 1m;
proxy_connect_timeout 5s;
proxy_read_timeout 10s;
health_check interval=5s fails=1 passes=1 uri=/ match=statusok;
}
# redirect server error pages to the static page /50x.html
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
access_log logs/es_access.log combined;
}
server {
listen 8080;
root /usr/share/nginx/html;
location = /api {
api write=on;
# directives to restrict access to authorized networks or users
}
location / {
index dashboard.html;
}
}
Summary
Elasticsearch, a powerful and flexible search engine, and NGINX Plus, an enterprise‑grade application delivery platform, fit naturally together as components of a modern scalable web architecture. As software products, they provide the same features and functions wherever you install them: on bare‑metal servers, in virtual machines, in the cloud, or in containers.
To try out NGINX Plus with Elasticsearch for yourself, start your free 30‑day trial of NGINX Plus today or contact us to discuss your use cases.
Note: Since publishing this blog post we’ve added support for HTTP/2 in NGINX 1.9.5 and NGINX Plus R7.
HTTP/2 Approved, Source: IETF
Since the IETF announced last week that the HTTP/2 specification has been finalized and will be published as an RFC, many customers have asked about our plans to support the new protocol in NGINX Open Source and commercial NGINX Plus products. We’re pleased to announce that we plan to release versions of both NGINX and NGINX Plus by the end of 2015 that will include support for HTTP/2.
HTTP/2, based on Google’s experiments with SPDY, promises faster page loading without the need for the uncomfortable page optimizations (domain sharding, resource merging, image spriting) often used to overcome design limitations in HTTP/1.1. A faster Internet benefits everyone, but adopting HTTP/2 also presents challenges, costs, and uncertainties. Our goal is to help you minimize these obstacles wherever we can, so you can choose how and when you begin publishing your web applications and content over HTTP/2.
The goal of HTTP/2 is to enhance the end user’s web experience by improving speed and performance, the same goal that NGINX has always pursued. We’re excited to take the lead in supporting HTTP/2 for our customers. It builds on our position at the forefront of vendor implementations of SPDY, which the IETF adopted as the basis for HTTP/2 – nearly 95% of websites that use SPDY are running NGINX and NGINX Plus. We spent considerable time and effort refactoring large portions of our core software to achieve an optimized SPDY implementation that is practical and useful in real‑world deployments. Our experience with SPDY puts us in a prime position to provide a similarly optimized HTTP/2 implementation and to continue support the protocol as it develops.
HTTP/2 is just one of a number of very exciting additions we’re planning for NGINX and NGINX Plus this year. We always value your input on how we’re doing and how we can improve – reach out to us anytime via the blog, Twitter, or our mailing lists (English and Russian).
[Editor – As announced in the original posting of this blog, we also periodically request formal feedback in a survey; this year’s survey closed on March 30.]
In some recent blog posts, we’ve explained why we believe it’s crucial to adopt a four‑tier application architecture in which applications are developed and deployed as sets of microservices. It’s becoming increasingly clear that if you keep using development processes and application architectures that worked just fine ten years ago, you simply can’t move fast enough to capture and hold the interest of mobile users who can choose from an ever‑growing number of apps.
Switching to a microservices architecture creates exciting opportunities in the marketplace for companies. For system architects and developers, it promises an unprecedented level of control and speed as they deliver innovative new web experiences to customers. But at such a breathless pace, it can feel like there’s not a lot of room for error. In the real world, you can’t stop developing and deploying your apps as you retool the processes for doing so. You know that your future success depends on transitioning to a microservices architecture, but how do you actually do it?
Fortunately for us, several early adopters of microservices are now generously sharing their expertise in the spirit of open source, not only in the form of published code but in conference presentations and blog posts. Netflix is a leading example. As the Director of Web Engineering and then Cloud Architect, Adrian Cockcroft oversaw the company’s transition from a traditional development model with 100 engineers producing a monolithic DVD‑rental application to a microservices architecture with many small teams responsible for the end‑to‑end development of hundreds of microservices that work together to stream digital entertainment to millions of Netflix customers every day. Now a Technology Fellow at Battery Ventures, Cockcroft is a prominent evangelist for microservices and cloud‑native architectures, and serves on the NGINX Technical Advisory Board.
In a two‑part series of blog posts, we’ll present top takeaways from two talks that Cockcroft delivered last year, at the first annual NGINX conference in October and at a Silicon Valley Microservices Meetup a couple months earlier. (The complete video recordings are also well worth watching.)
This post defines microservices architecture and outlines some best practices for designing one.
Cockcroft defines a microservices architecture as a service‑oriented architecture composed of loosely coupled elements that have bounded contexts.
Loosely coupled means that you can update the services independently; updating one service doesn’t require changing any other services. If you have a bunch of small, specialized services but still have to update them together, they’re not microservices because they’re not loosely coupled. One kind of coupling that people tend to overlook as they transition to a microservices architecture is database coupling, where all services talk to the same database and updating a service means changing the schema. You need to split the database up and denormalize it.
The concept of bounded contexts comes from the book Domain Driven Design by Eric Evans. A microservice with correctly bounded context is self‑contained for the purposes of software development. You can understand and update the microservice’s code without knowing anything about the internals of its peers, because the microservice and its peers interact strictly through APIs and so don’t share data structures, database schemata, or other internal representations of objects.
If you’ve developed applications for the Internet, you’re already familiar with these concepts, in practice if not by name. Most mobile apps talk to quite a few backend services, to enable their users to do things like share on Facebook, get directions from Google Maps, and find restaurants on Foursquare, all within the context of the app. If your mobile app were tightly coupled with those services, then before you could release an update you would have to talk to all of their development teams to make sure that your changes aren’t going to break anything.
When working with a microservices architecture, you think of other internal development teams like those Internet backends: as external services that your microservice interacts with through APIs. The commonly understood “contract” between microservices is that their APIs are stable and forward compatible. Just as it’s unacceptable for the Google Maps API to change without warning and in such a way that it breaks its users, your API can evolve but must remain compatible with previous versions.
Best Practices for Designing a Microservices Architecture
Cockcroft describes his role as Cloud Architect at Netflix not in terms of controlling the architecture, but as discovering and formalizing the architecture that emerged as the Netflix engineers built it. The Netflix development team established several best practices for designing and implementing a microservices architecture.
Create a Separate Data Store for Each Microservice
Do not use the same backend data store across microservices. You want the team for each microservice to choose the database that best suits the service. Moreover, with a single data store it’s too easy for microservices written by different teams to share database structures, perhaps in the name of reducing duplication of work. You end up with the situation where if one team updates a database structure, other services that also use that structure have to be changed too.
Breaking apart the data can make data management more complicated, because the separate storage systems can more easily get out sync or become inconsistent, and foreign keys can change unexpectedly. You need to add a tool that performs master data management (MDM) by operating in the background to find and fix inconsistencies. For example, it might examine every database that stores subscriber IDs, to verify that the same IDs exist in all of them (there aren’t missing or extra IDs in any one database). You can write your own tool or buy one. Many commercial relational database management systems (RDBMSs) do these kinds of checks, but they usually impose too many requirements for coupling, and so don’t scale.
Keep Code at a Similar Level of Maturity
Keep all code in a microservice at a similar level of maturity and stability. In other words, if you need to add or rewrite some of the code in a deployed microservice that’s working well, the best approach is usually to create a new microservice for the new or changed code, leaving the existing microservice in place. [Editor – This is sometimes referred to as the immutable infrastructure principle.] This way you can iteratively deploy and test the new code until it is bug free and maximally efficient, without risking failure or performance degradation in the existing microservice. Once the new microservice is as stable as the original, you can merge them back together if they really perform a single function together, or if there are other efficiencies from combining them. However, in Cockcroft’s experience it is much more common to realize you should split up a microservice because it’s gotten too big.
Do a Separate Build for Each Microservice
Do a separate build for each microservice, so that it can pull in component files from the repository at the revision levels appropriate to it. This sometimes leads to the situation where various microservices pull in a similar set of files, but at different revision levels. That can make it more difficult to clean up your codebase by decommissioning old file versions (because you have to verify more carefully that a revision is no longer being used), but that’s an acceptable trade‑off for how easy it is to add new files as you build new microservices. The asymmetry is intentional: you want introducing a new microservice, file, or function to be easy, not dangerous.
Deploy in Containers
Deploying microservices in containers is important because it means you just need just one tool to deploy everything. As long as the microservice is in a container, the tool knows how to deploy it. It doesn’t matter what the container is. That said, Docker seems very quickly to have become the de facto standard for containers.
Treat Servers as Stateless
Treat servers, particularly those that run customer‑facing code, as interchangeable members of a group. They all perform the same functions, so you don’t need to be concerned about them individually. Your only concern is that there are enough of them to produce the amount of work you need, and you can use autoscaling to adjust the numbers up and down. If one stops working, it’s automatically replaced by another one. Avoid “snowflake” systems in which you depend on individual servers to perform specialized functions.
Cockcroft’s analogy is that you want to think of servers like cattle, not pets. If you have a machine in production that performs a specialized function, and you know it by name, and everyone gets sad when it goes down, it’s a pet. Instead you should think of your servers like a herd of cows. What you care about is how many gallons of milk you get. If one day you notice you’re getting less milk than usual, you find out which cows aren’t producing well and replace them.
Netflix Delivery Architecture Is Built on NGINX
Netflix is a longtime user of NGINX Open Source and became the first customer of NGINX, Inc. after it incorporated in 2011. Indeed, Netflix chose NGINX as the heart of its delivery infrastructure, Open Connect, one of the largest content delivery networks (CDNs) in the world. With the ability to serve thousands, and sometimes millions, of requests per second, NGINX and NGINX Plus are optimal solutions for high‑performance HTTP delivery and enable companies like Netflix to offer high‑quality digital experiences to millions of customers every day.
Video Recordings
Fast Delivery
nginx.conf 2014, October 2014
Migrating to Microservices, Part 1
Silicon Valley Microservices Meetup, August 2014
Migrating to Microservices, Part 2
Silicon Valley Microservices Meetup, August 2014
NGINX is excited to sponsor SCALE 13x (Southern California Linux Expo)! SCALE is the largest community‑run FOSS conference in North America. NGINX is proud to join other top industry experts talking about open source.
Come meet the team and learn why NGINX is the secret heart of the modern web. We’ll show you how to improve application performance quickly and easily with NGINX.
Attend Sarah Novotny’s talk on Saturday, Feb. 21 @ 16:30: NGINX 101 – now with more Docker. Learn the three most basic NGINX use cases of static content delivery, application load balancing, and web proxying with caching; all within Docker containers.
Stop by NGINX booth 79. Meet technical experts and learn how to improve application performance quickly and reliably. You could even get your very own NGINX t‑shirt.
Join us for the Weakest Geek on Saturday, Feb. 21 @ 20:00. Come see members of the free & open source community – including Sarah Novotny from NGINXnbsp;– face off against each other, determining who has the greatest knowledge of geek culture. Learn more.
Today’s businesses face disruption and opportunity on a scale we have never seen before. As developers and technology professionals, we find ourselves at the center of this storm. What sets top development teams apart from the rest is the ability to innovate and adapt, to build and deliver extraordinary new products and experiences faster than the competition.
“Building a great application is only half the battle. Delivering your great application is the other half.”
No matter how sophisticated, functional, and beautiful a great application may be, if it does not perform in the real world, for real users, it has not succeeded.
Why Is Delivering An Application So Hard?
Real‑world deployments subject an application to stresses and strains it will never encounter in development:
Slow client traffic ties up server resources and impacts performance in a way that’s extremely difficult to measure in the lab.
Unpredictable spikes in traffic can overwhelm the application platform.
Search engine, screen‑scraping, and attack traffic can consume resources uncontrollably, and may leak sensitive data.
Multiple client types over multiple network types can defeat attempts to optimize the application stack.
Modern acceleration and security processes may not be well supported by the application framework.
The simple problem is this: modern application platforms (like Ruby‑on‑Rails, Grails, Python/Django, or even J2EE) are engineered for developer productivity first, not performance or scalability. What can you do to harden and prepare your application for the onslaught of traffic it will encounter in production?
How Can I Make My Application Production‑Ready?
There are a great many best practices and steps that you can take to make an application production‑ready. Caching, load balancing, HTTP optimizations, request scrubbing, and offloading operations like compression and encryption – these all contribute to improved performance, resilience, and control:
Caching reduces application load and delivers content faster to end users.
Load balancing distributes traffic across application instances to optimize performance and avoid failed or underperforming servers.
HTTP optimizations transform slow client traffic into fast local transactions so that applications operate at peak efficiency (as they do in a benchmark).
Request scrubbing cleans up traffic and drops bad requests before they can impact the application.
Compression and encryption offload takes work from application servers so that they can focus on what they do best – running your application!
NGINX is a lightweight, highly reliable reverse proxy that is deployed as a web accelerator and web server in front of your infrastructure, and NGINX Plus extends this with more sophisticated load balancing, application delivery and monitoring capabilities. It is proven (with over 10 years of production deployment), it’s scalable (uses minimal memory and CPU), and it’s extremely fast. It is often deployed as the first tier for user traffic for the following reasons:
Single entry point for multiple applications – Terminate and route traffic (HTTP, WebSocket and TCP‑based protocols) to multiple web applications, concealing the internal application infrastructure from external clients. The configuration language makes request routing easy, with a structured format that breaks out traffic by application (Host header) and URL.
Load balancing, visibility, and health checks – Distribute incoming traffic evenly across multiple application instances, informed by the load‑balancing policy, performance measurements, and health checks. The net result is that you optimize application performance by diverting traffic away from failed and underperforming servers.
Caching – Use the massively scalable hybrid disk and memory store for HTTP caching. The cache reduces the load on the application by serving duplicate requests, and speeds delivery to end users by responding directly to these requests. Even for fast‑changing applications like news feeds and blogs, you can reap enormous performance improvements by caching for a couple of seconds at a time (microcaching).
Static content offload – Move static content to the NGINX or NGINX Plus front end and serve it directly. It’s is not just an accelerating proxy, it’s also a full‑fledged web server. You can perform speculative checks to see if the content is present locally (the try_files directive) and forward the request to an upstream server if not.
You can regard NGINX and NGINX Plus as a shock absorber, protecting your applications from rough and unpredictable Internet traffic, giving the smooth ride that enables them to perform at their best.
What’s The Next Step?
NGINX Plus does not just help you to harden your application, readying it for production traffic. When you scale your application up from one development instance to multiple production instances, NGINX Plus is the one constant element that gives you control.
“All problems in computer science can be solved by another level of indirection.”
Wikipedia defines indirection as “the ability to reference something using a name, reference, or container instead of the value itself”.
With NGINX Plus, users reference your application instance through NGINX Plus rather than directly. NGINX Plus is a level of indirection that gives you control:
Faster deployment – After you provision your application, adding a routing rule (and optional authentication) to the NGINX Plus traffic management tier is all you need to do to make it available and to start monitoring traffic.
A/B testing – Using NGINX Plus’ split_clients directive, you can safely test alternative application versions, siphoning off a small proportion of your users to a new version so you can conduct side‑by‑side tests.
Seamless application upgrades – When deploying a new instance of your application, you can use NGINX Plus to gradually move users across as their sessions complete, without any interruption in service or loss of traffic.
Taking a server out of service – You can instruct NGINX Plus to stop sending new requests to an application instance that you are removing from an upstream group, but let existing sessions complete. This is known as ‘draining‘, and provides a safe way to take servers down without interrupting or impacting user sessions.
NGINX Plus is the trusted gatekeeper, giving you full control over which traffic you admit to your applications, how it is rate‑limited, authenticated, and secured, and where it is routed to.
Finally, Transforming Your Applications For The Future
Organizations are navigating their way through complex application changes. The desire to optimize for speed of delivery rather than cost is driving significant transformations in the ways that development teams build modern web applications.
“Nordstrom went from optimizing for IT cost to optimizing for delivery speed.”
Your application infrastructure and your development processes will suffer great upheavals. Processes will change when you adopt continuous delivery and DevOps instead of big‑bang waterfall‑driven releases. Architectures will change when you adopt microservices instead of monolithic web applications. Deployment will change when organizations adopt cloud‑based as‑a‑Service platforms and reduce their dependencies on on‑premises software and hardware infrastructure.
In the face of this continual upheaval, you’re expected to deliver consistently high‑quality applications. NGINX Plus is the one constant you can count on. No matter what happens “behind the scenes,” no matter where you deploy your applications or the languages and platforms you select, NGINX Plus is the consistent, reliable, and high‑performance application delivery platform that you, and millions of other developers, rely on.
You build great applications. NGINX delivers them.
And try out our fully supported, enterprise‑grade application delivery platform, NGINX Plus – start your free 30-day trial today or contact us to discuss your use cases.
This is a guest post by our friends at MaxCDN, and was originally posted on their blog.
NGINX is a high‑performance web server, load balancer, and proxy that our CDN network uses to deliver our customers’ content as fast as possible.
In the past few years, more high‑traffic websites have started using NGINX as their high‑performance server. Basically, the popularity Apache started losing in 2011 has been picked up by NGINX, making it today’s most popular web server for the top 1,000 busiest websites. NGINX was also growing steadily at 2% when Netcraft conducted its August 2014 Web Server Survey.
Here’s an illustration of the study’s main findings:
We don’t just use NGINX because it’s popular, though. We use it because it’s event‑driven, effectively handles large traffic spikes, and helps us scale our network. It also helps us make cool $#!+ and offer next‑gen performance features to our customers.
Key Features that Make NGINX Cool
Key components of NGINX include:
Multiple protocol support (HTTP, HTTPS, WebSocket, IMAP, POP3, SMTP)
SSL termination (TLSv1.1/TLSv1.2/SSL/SNI/PFS/PCI‑DSS and OCSP stapling)
HTTP video streaming using MP4/FLV/HDS/HLS
Caching static and dynamic content
Content compression
Header manipulation
Request filtering
Extended monitoring and logging
Upgrades without downtime using live binaries
Graceful restart with nonstop request processing
Complete reverse proxy and load balancer
These components team up to power our network and quickly deliver all the content cached on it. They also complement all the NGINX awesomeness mentioned below.
Rock‑Solid Architecture
NGINX and Apache are the most used open source web servers. However, for a long time, system administrators have always used Apache as the default web server for their online projects. This became a problem though when they were working on projects that needed to handle big traffic. Apache is a solid server, but isn’t necessarily good for handling tons of traffic efficiently.
This is where NGINX rises to power though. Using its advanced architecture illustrated below, NGINX outperforms Apache in terms of RAM, CPU, and number of requests served. It is also event‑driven unlike traditional servers. This means it doesn’t need to spawn new processes or threads for each served request, resulting in lower levels of memory usage.
NGINX now ranks as the second most used open source web server. It’s also great for all kinds of operating systems like Linux, even Windows. Without a doubt, it’s a smart choice for developers from all platforms, and we especially love it because of our dedication to open source.
Outstanding Performance
RAM and CPU usage are incredibly small given the tons of content this beast can handle. Videos, images, even dynamic pages can be served without any out‑of‑the‑box tuning. It’s fair to say that NGINX offers 3x better performance than traditional servers.
Highly Scalable
Native built‑in load balancing and proxy capabilities, plus the way its core was designed, makes it perfect for scalability. Because scalability is an essential part of our CDN, it’s much easier to bring our CDN to scale with NGINX on our side.
Extended Documentation
If you have any questions about how to use NGINX, how to install it, or how to configure it, there’s a big community behind the server that can help you get the answers to your most pressing questions:
See how MaxCDN and NGINX help you deliver your content as fast as possible – download our whitepaper. It shows you how we brought our CDN to scale and became one of the fastest CDNs in the world.
It wasn’t long ago that open source software was the poor cousin of proprietary tools. It was the software you picked if you couldn’t afford “real software” and were willing to make do with fewer features and spotty support.
Not anymore.
Instead of asking “why open source?”, CIOs now first ask “where else can we use open source?” Open source software has emerged as the leader in many categories and the driving force of technology innovation, from cloud and big data to social media and mobile. NGINX believes that the broad adoption and contribution to open source represents the greatest opportunity for continued innovation and economic growth in our digital-centric society.
This is why we are proud to be platinum supporters of this year’s Future of Open Source Survey, sponsored by Black Duck and North Bridge Venture Partners. This annual assessment of open source industry trends drives broad discussion around key issues for developers, technology purchasers, and the open source community at large. Continue reading “NGINX Supports Annual Future of Open Source Survey”›
The new year is well underway and, like many others, you are probably planning upcoming projects to enhance your existing digital experiences or build completely new websites and applications. As you do so, the most critical fact to keep in mind is that “web” no longer means “web browser.” Pundits have been espousing a “mobile first approach” to application development for years, but with an annual mobile traffic growth rate of nearly 60% this is not negotiable any longer. The explosion in the number of connected devices, the shift of traffic from desktop to mobile, and the rising tide of the internet of things means that projects must now be rooted in the diversity of user experiences available on different devices.
It’s becoming increasingly obvious that the historical approach to application architecture – we’ll call it the monolithic model, due to the focus on building a single integrated application containing the majority of features and functions – starts to break down in a world where you need to tailor your experiences for specific devices and specific users or use cases. Moreover, you simply cannot achieve the agility, flexibility, and scalability demanded of today’s applications using many of the old architectural patterns and behaviors.
![IETF approved HTTP/2 [image]](https://cdn.wp.nginx.com/wp-content/uploads/2015/02/IETF_HTTP2_Approved-300x209.png)
Fortunately for us, several early adopters of 
