Deis, DevOps and the Future of Open PaaS

These are exciting times in web and mobile development. Innovations in deployment automation and infrastructure orchestration are coming hard and fast with no sign of slowing. You might say there's never been a better time to be a developer.

Those of us focused on making developers’ lives easier have gravitated toward the "Platform as a Service" model. It turns out you can keep developers happy and productive by providing them an application platform (PaaS) that neatly decouples their applications from the operating system.

Today, most automation-minded organizations are still investing in and succeeding with server-based application deployment. However, now that LXC has begun to demonstrate security and efficiency gains reminiscent of modern virtualization, the era of Open PaaS is here.

Heroku and Google App Engine have proven that application platforms based on Linux Containers can scale. Now the race is on to create open source alternatives that provide this application platform capability inside the enterprise.

DevOps vs. NoOps

The "NoOps" movement presumes operations teams will be made obsolete by highly automated application platforms. Hosted PaaS solutions nearly eliminate the need for DevOps skill-sets, lending credence to NoOps. However, the reality is today's public application platforms can be shockingly expensive for even moderate workloads. Worse yet, the very abstractions they promote are often exactly what frustrate sophisticated teams.

These frustrations eventually lead to a search for a "private Heroku" or "private App Engine" that offers the same benefits for developers, but without the loss of operational control.

Unfortunately, the Open PaaS solutions available today (Cloud Foundry and OpenShift) introduce wholly new paradigms and components that, while ostensibly open source, are in practice difficult to install, operate and customize. This presents an opportunity for an open source application platform that can bridge the gap between the DevOps tool chains trusted by today’s sophisticated ops team and our containerized future. Deis is that platform.

Introducing Deis

Deis combines Docker's Linux container engine with Chef’s trusted infrastructure automation, creating an open source application platform that runs on public cloud, private cloud and bare metal. By building upon Chef, which is widely deployed and proven to work at scale, Deis provides a "private Heroku" experience that delights both developers and operations engineers.

The Power of Docker

Having been involved with the Docker community since their inception, we've had a front-row seat to their meteoric rise and have great respect for the team and technology. Docker is one of those rare innovations that seem so obvious in retrospect. Portable application containers based on LXC, file system de-duplication and tooling to automate building, distributing and executing containers. Obviously! The trick is in making these "standard containers" ubiquitous.

While the full promise of Docker has yet to be realized, the team’s ability to capture open source momentum toward Linux containers is impressive. The project is approaching 200 external contributors, featuring some large contributions including how Docker interacts with host volumes and Docker's file system layering technology. This ability to harness open source momentum is what makes us believe Docker will succeed in its mission to standardize application containers across the modern datacenter. As a leading Docker PaaS, Deis will continue to play a role in this effort.

Evolution Toward Chef Server

Prior to Deis, the OpDemand team wrote a lot of Puppet. We watched as the industry began moving toward Chef, with RightScale and AWS OpsWorks demonstrating support. Chef was quick to develop a thriving DevOps community that produced hundreds of high quality cookbooks and a first-class tool chain (knife, berkshelf). We decided to try it.

It wasn’t until we built Deis that we fully appreciated the flexibility of Chef's Ruby DSL, the determinism of explicitly ordered resources (in contrast with Puppet’s frustratingly indirect RAL) and the reliability of Chef Data Bags as a global store for PaaS configuration. Without a Chef Server, Deis becomes significantly more complicated and brittle.

If someone can demonstrate a faster, more reliable, and more flexible approach to configuration management than Chef Server, the Deis project is open to it. Deis sports a pluggable configuration management interface, and we will gladly accept a well-designed pull request. For now, Deis is happy to be part of the healthy Chef ecosystem.

Heroku Inspired

When looking for inspiration in the world of PaaS, you can't ignore Heroku. The command line workflow they've pioneered is second to none. By treating the developer community as artists, they've managed to build a loyal following – one we got to see up close at their Waza conference earlier this year. If your goal is to delight developers, emulating Heroku is a great place to start.

While Deis was designed with Heroku's Twelve Factor methodology and command line workflow in mind, Heroku has very little to say about the guts of the application platform and runtime. This is where Deis and Heroku diverge. Deis introduces the concepts of Formations, Layers, and Nodes to help operations teams deploy a private application fabric built to their own specifications using Chef and Docker.

Although we disagree with their NoOps stance, Deis will continue to take inspiration from Heroku as we strive to become the best Open PaaS on the market.

Future of Deis

Deis has only been available for two months. The open source community's reaction has been exciting and humbling. We have a lot of work to do on the installation process, backing services, system hardening, access controls, load testing and more before we can make our claim as the enterprise-grade "private Heroku" – but we're damn close. With the help and support of the open source community, we expect to announce our 1.0 release in a few months. Until then, expect a steady stream of releases featuring bug fixes, new cloud providers, enhanced Docker integration and much more.

If you're interested in learning more about Deis or contributing to the project, come find us in IRC (#deis on Freenode) and check out the project on GitHub.

24 Oct 2013

Gabriel Monroy

Gabriel Monroy

Cloud operations nerd and CTO at OpDemand

Follow me on Twitter @gabrtv