VMworld 2017: Pivotal Container Services (PKS)

IMO – this is one of the biggest announcements at VMworld this week.   It is a major shift to the Dell Technologies strategic perspective, and important for our customers.

It’s something we’ve been spending a lot of time on internally for a while – I’ve spent hours at end with Scott Yara and James Watters at Pivotal, and with Ray O’Farrell and Paul Fazzone at VMware, and it’s not just me, but others in Dell EMC too.

Let’s start from the start.  There are 4 abstractions customers need as they develop cloud native apps:

Each has a purpose – one that the others are not ideal for.

• Kernel-Mode VM =   an instantiation of an abstraction of a physical host.  Useful when you need to get low-level for one reason or another, including hardware abstraction.   It’s notable that almost all containers run on some form of kernel mode virtualization.   Things like hardware abstraction, security abstraction, software defined networking are insanely valuable even when the sole purpose of the VM is to host containers.   Yes, in the 100% container scenario some functions like VM consolidation, HA, resource scheduling have reduced value – but those other things = valuable.
• Container/Cluster Manager = an instantiation of an abstraction of a OS including a namespace, and meta filesystem.   Useful when you can’t use a structured PaaS, but need your own specific set of tools, functions, runtimes, micro-services – in other words, build your own DIY PaaS on top of the container/cluster management layer.
• A structured PaaS = an instantiation of an abstraction of a given set of runtimes and given platform services.  Useful when you want a turnkey platform and it’s a fit for what you need.
• Data Services and persistence = an instantiation of given set of data services (like Dynamo, RDS, but also SQL, Redis, you name it) and object/blob storage.  Useful when you need to store/pricess stateful information.

The preference is always to use the highest order abstraction you can…  that means you need to do less, and can focus on things that have value.

Now – as Dell Technologies – we’re already doing great when it comes to 3 out of 4.

• VMware has the best Kernel Mode abstraction platform on the planet.  Mature.  Widely deployed.  Feature rich.
• Pivotal is the leader in structured PaaS abstractions with Pivotal Cloud Foundry.
• Dell EMC has great object storage abstraction models for developers with Elastic Cloud Storage – and Pivotal Data services is a rich set of data service abstractions of all types.

We all collectively participate and contribute (and use) the container ecosystem – but didn’t have a strong opinionated point of view.

• Dell EMC {Code} makes tons of contributions to volume controllers into Docker, Mesos, Kubernetes. 
• Pivotal make Docker a choice underneath Diego. 
• VMware integrates Photon platform elements and vSphere Integrated Containers (VIC) with the whole ecosystem.

But… historically, our point of view on the container/cluster manager abstraction ecosystem wasn’t clear.

I’ve heard some folks vehemently and passionately argue that container/cluster managers aren’t needed (“just use a structured PaaS!”.   I’ve heard others argue that developers don’t need Kernel Mode VMs.  I’ve heard others claim infrastructure doesn’t matter. 

Hint – none of those are correct.  There is a lots of confusion…. and that’s not a surprise, this is a VERY dynamic/chaotic ecosystem right now.

The container/cluster manager ecosystem seems to be starting to settle.

From my point of view, and it’s a broadly held point of view in Dell Technologies – Kubernetes is the likely “winner”.   It seems that it’s hit critical mass – and enough people are rallying around it – just like people have rallied around Cloud Foundry as the structured PaaS of choice.

The strategic pivot was to acknowledge that we needed to do our part, and that meant a couple things:

1. Make our point of view clear – anyone inside the company that thinks there isn’t a need for container/cluster managers is not aligned with our strategy.   PCF is great – but it isn’t the answer for need of a developer, of a customer building new cloud native apps.
2. Continue to be open to the container/cluster ecosystem, but have a strong opinion.  We do – Kubernetes.
3. Find the way where we can best contribute and help.    The answer is around contributing, hardening, and generally making Kubernetes enterprise-ready in every way we can.

Google is the primary contributor to Kubernetes – and saw the great opportunity of us adding to the community – and thus Kubo was born.   Kubernetes on BOSH – the same very bottom level part that underpins Pivotal Cloud Foundry.   BOSH performs an analagous function to the BORG layer within google itself – a low-level lifecycle management and physical resources scheduling function.

This is an area where we have a lot to give.  We can make core Kubernetes services liked etcd easier to deploy, to lifecycle.  We can build integrations with the VMware Cloud Foundation stack that make Kubernetes stronger and better – and we announced material resources working on Photon at VMware shifting to focus there.   We can take the concepts of Developer-Ready Infrastructure and NSX/PCF integration and extend that to the Kubernetes ecosystem.   Dell EMC can integrate ECS more tightly.   We can build and integrate object snapshot/versioning schedulers and managers as a “backup” analog for cloud native apps.

Personally, the team I lead can work to make VxRack SDDC and VxRail the best way, hands down, to deploy this stack on premises.

Today – Pivotal, VMware, and Google announced Pivotal Container Services – which has an acryonym of PKS – with the K signifying Kubernetes.   Pivotal Container Services is the enterprise distribution of Kubo.

Here is is in a simple picture I’ve sketched up:

There are some key observations here:

• We will now give developers – all four abstractions, and leaders in each.
• While we continue to be open – we’re now strongly aligned and opinionated on the best way to do this – something the whole of Dell Technologies can and will rally around:
• Pivotal Cloud Foundry is the most mature, most deployed structured PaaS.
• Pivotal Container Service is the best Kubernetes based container/cluster manager for the enterprise.
• VMware stack is the most mature kernel mode virtualization stack on which to build in the enterprise, and it’s one you already know.
• Dell EMC VxRack SDDC (build on VMware Cloud Foundation and extended with additional services like ECS) and VxRail (for customers who want to start really small) will be the best turnkey HCI optimzed for the full stack.
• We know that not every customers will want/need it all.  Some will be OK with just PCF.  Other OK with just PKS.  We will not connect the licensing model – ergo, while PKS shares an architectural underpinning with PCF in BOSH, a customer doesn’t need PCF to use PKS.

BTW – this is one of the elements behind the Pivotal Ready System announcement I talked about here.

What you have today is a cool technical announcement, a sign that Pivotal, VMware, Dell EMC are embracing Kubernetes.

But – as a leader in the company, I also see something else. 

We now have a Cloud Native/Digital Transformation stack where there is a SINGLE target we are furiously running towards now as VMware, Pivotal, and Dell EMC – no mis-alignment, no differences in PoV. 

Today you see the hand of the leadership of the new Dell Technologies company (Michael Dell) and the CEOs of the strategically aligned companies (VMware – Pat Gelsinger & Pivotal – Rob Mee) all saying: “we’re going THIS way, come with us”.

Lots of exciting work going on here – and I’m pumped to finally get to talk about it!