Christopher Litsinger, Director, Cloud Application Platforms at Comcast
Back in 2015, a small, scrappy group of engineers set out to replace a commercial API management gateway with an in-house solution based on NGINX, then converted the sytem to self-service. A year later they emerged, singed but triumphant, with hundreds of existing services transparently migrated to the new platform. Latencies were reduced and acclaim rang out: “The old 50th percentile is the new 99th!” clients said with glee.
The improvements drove unforeseen growth. Over two years, the number of hosted services more than doubled. Data centers came and went. Thousands of new credentials were added to the system. Transaction rates that once peaked at 2,000 RPS grew to 33,000. The small DevOps team, in its rush to stop paying for a commercial solution, had made a strategic decision to hold off on delivering self‑service. Now they had no choice but to prioritize the manual labor required to add services and credentials while working towards self‑service, before eventually delivering self‑service to its consumers.
In his session at NGINX Conf 2018, Christopher unpacks events as they unfolded, looks at the decisions that were made, and describes the lessons learned from the challenges his team faced.
