Last week we published a huge testimonial from a great customer (Verizon) about what we’re doing with them around their network function virtualization and cloud efforts. Hint, it doesn’t center around storage arrays. It’s about Mesos and Mesosphere DCOS, CoreOS – and it’s all software on commodity-off-the-shelf hardware using EMC ScaleIO and Elastic Cloud Storage (object storage). Don’t listen to me – listen to the customer – here.
I’m disturbed that many customers view us as the “VMAX/VNX/DataDomain” company – those are great platforms, and power a TON of the world’s primary and backup storage. BUT – that mental image that “brand identity” of a “big iron external storage vendor” is incomplete. Sure, we do that (and do it well) – but we are much more. We do converged and hyper-converged infrastructure, we do SDS, we do material open-source contributions, and in some cases, lead OSS projects.
I was personally challenged on this front (“brand identity”) recently. A great customer (Canadian Tire – thanks for being a customer!) visited us in Marlborough who challenged us about whether this embrace of open source is new for us – and whether we were in it for the long haul (P.S. CT team – still fingers crossed to get the ardunio IoT christmas lights :-)
Answer – about 2 years ago, we made a strategic pivot where we recognized that winning in these new “platform 3”/”mode 2” cloud native app domains requires that you completely embrace open source models. In other words (and lifting metaphors that are well-worn), it’s not about free beer, but free speech. it’s not about commercials – it’s about the community either embracing or rejecting what you’re doing.
That pivot 2 years ago takes time to manifest as real projects, real code, real contributions – but that’s in full flight now. And this week’s news is more on that front.
… For the community to embrace you in the Cloud Native domain – it’s all about the code, it’s all about quality. There’s a natural suspicion of vendors who have large “incumbent” businesses which have historically been closed source or proprietary appliances.
I’m sure that some are “faking it” – but I can tell you that we are not. Now - whether we win or lose has yet to be proven. Success will be up to execution, solid code, and embrace of new business models that yes, disrupt ourselves – but we are swinging for the fence, in the same way we’re disrupting ourselves with new economic consumption models (Virtustream), converged and hyper-converged technology consumption models (Vblock/VxRack), and our traditional storage base (with XtremIO, with ScaleIO, and with cloud target integration in CloudArray, CloudBoost, Cloud Pool).
Today is another page turning in this open-source journey. In recent chapters:
- In November, I noted what we were doing around OpenStack here.
- Then, I noted what the gang at Pivotal was doing around making the Hadoop-Native SQL engine, and the world first open source MPP database for DW here. This completed Pivotal embracing open source models for their entire stack – on the PaaS side, Cloud Foundry and Spring, and on the data side, the Open Data Platform (ODP) including open-sourcing their leading SQL transactional engine, in-memory grid, and MPP database under Apache 2.0.
- Also in August at VMworld VMware launched the Photon Platform (here), and November, the VMware crew came out swinging with the code for the Photon controller here (and there’s a deep-dive blog post here). The Photon Platform for their next-generation Cloud Native IaaS stack for customers who love vSphere and VMware, but want something lighter, something designed for high scale, high churn – and thinks of containers as the “first order abstraction” vs. kernel-mode VMs. Lots of activity and momentum behind making the photon platform be as compelling for pure cloud native apps as the vSphere based SDDC stack is for workloads that need resiliience
Today:
- EMC is releasing RackHD (“Rack Hardware Director”) – an open source project that has the goal of making low-level hardware management (VERY low level) at rack and hyper-scales easier. It covers hardware management and orchestration (M&O) that automates discovery, description, provisioning and programming across a broad range of servers today and a roadmap to add networking devices in the future. Think of how hard it is to do low-level functions like firmware updates across hardware vendors, or telemetry, or pre-boot configuration. Yes, there are tools to do this – but they are generally vendor-specific and closed. We are working to RackHD is an important part of our rack and hyper scale hyper-converged offer: VxRack (RackHD is the lower levels of VxRack Manager). There’s even an early-stage CPI (Cloud Provider Integration) prototype for Cloud Foundry to bolt right onto RackHD for customers who want Cloud Foundry on bare metal. RackHD is released under the Apache 2.0 license. Code speaks: https://github.com/RackHD
- BTW – last week Intel released some important code to support their vision of Intelligent Resource Orchestration (IRO) - with a focus on telemetry (more here). Congrats to Nick Weaver and the whole Intel SDI team! Intel is a huge partner – and perhaps the largest driving force in this space. We’ve aligned RackHD with the overarching “Rack Scale Architecture” (RSA) efforts at Intel – think of RackHD as a “reference implementation” of the RSA model. Furthermore, we’re going to leverage the IRO plugin framework to accelerate our telemetry capabilities which we were going to build ourselves into RackHD otherwise. This is about giants aligning, and contributing.
- EMC is releasing CoprHD 2.4 – an open source project that has the goal of being an abstraction, pooling and automation layer for persistence in ALL forms (transactional, NAS, Object, HDFS), and doing it in a way that is open both northbound (orchestrators) and southbound (persistence/storage targets). ViPR Controller is the “enterprise distribution” of CoprHD – and yes, when we launched the ViPR Controller in 2013, we knew that it needed to be an open source project to ultimately be all it could be. CoprHD 2.4 adds support for XtremIO and Elastic Cloud storage – but more importantly, is the first release with major non-EMC contributions from Intel, Oregon State University and others. Like RackHD (and you can see a pattern here), CoprHD 2.4 has switched to align with the Apache 2.0 license. Code speaks: https://coprhd.github.io/
- EMC delivers big updates to Project REX-Ray. The EMC{code} team has been cranking, and this is moving fast, and becoming a great (best way?) to integrate transactional persistence models (versus restful object models) into container runtimes, management frameworks and cluster managers like Docker, Mesos, and others – without breaking the idea of ephemeral container models that never contain configuration. REX-Ray 3.0 adds storage platform support for GCE (Google Compute Engine) as well as EMC Isilon and EMC VMAX in addition to the already existing ScaleIO and XtremIO support. Also, there is now a a pre-emptive volume mount function that enables the host to reassign mounted volumes from non-responsive hosts. This ensures applications maintain access to persistent storage in the crazy dynamic world of ephemeral containers :-) And yes, of course, we have prototypes that integrate with the VMware Photon Controller and Photon Machine layer. REX-Ray is released under the Apache 2.0 license. Again code speaks - the bits will be posted on Dec 12th: https://github.com/emccode/rexray.
One day – 3 big open source releases – and over the last month, another 3 – and that’s just from VMware, EMC, Pivotal.
That is of course a drop in the bucket in the world of Open Source projects... Apple open-sourced Swift yesterday. There was a huge Puppet release in recent weeks. There was a major set of Spring updates over the last month. Tons going on in the Apache Hadoop projects (neat to see HDFS get erasure coding). The Open Container Initiative (OCI) and Dockercon along with Mesoscon in November. Kubernetes keeps trucking along. The OpenStack summit and Liberty release continue to harden.
Now – for readers whose heads are spinning, and feel “my goodness, all these new words, new tech, and new business models – how do I keep up with all this stuff?” - don’t feel alone :-)
I’m finding that in my day job, working to keep the EMC Systems Engineering teams (who are great) sharp on all this is a trick, and strains their brains (and mine), and how we train and enable each other.
I think that inevitably (and I could be wrong here), that the job of the enterprise ecosystem will continue to be to simplify, package, deliver, and support this market.
Very few customers I see pull from the open source trunks – and instead partner with vendors and integrators that are: a) committed; b) committers; c) competent; d) have a distinct and transparent point of view.
Going into 2016 – I think what I (and more broadly EMC) can do is continue to do A-D in the list above – and simplify and “curate” and “industrialize” these stacks… Ultimately customers don’t care about the tech itself, only the outcome they deliver. That means we need to keep “industrializing” a rich, robust, resilient IaaS stack (vSphere/vRealize), a lightweight, API driven, high churn Cloud Native IaaS/PaaS stack (stay tuned – but you can see us getting ready and building up muscles), and the data analytics ecosystem (ODP / Cloudera / Splunk / etc).
I think we need to keep making this simple – and driving towards converged infrastructure (Blocks/Racks/Appliances) as the hardware strata under those stacks – and deliver them as CAPEX or OPEX, on premises and off-premises (this is what Virtustream is really about) – and any mixture of the above.
It’s a tall task – but a noble one, an exciting one, and a task that I think we can uniquely do – particularly as Dell and EMC come together.
Now, strategically, we get that you can’t fake open – and that people will only partner on “industrialized” stacks from partners they trust.
I’m going to keep doing everything in every fibre of my being to keep driving down that path – but it’s not about me, there are a ton of people in the company working down that path. For them, and for you, today is a big milestone!
Would love your feedback on the strategic direction, but just as much – do a github clone of the projects, and give ‘em a whirl!!!
