SDS and HCI Part 2: Two SDS Approaches

In Part 1 of this series, I covered the “why” behind the fact that SDS and HCI models are the defecto “start” for me for most customers and most workloads (and not only for me, but increasingly for Dell Technologies as a whole).   BTW – the survey results from Part 1 are now available here.

Next question: are all SDS models the same?

In one sense – sure. They are a software storage/persistence stack that has no hardware dependency.

Beyond that – there’s a ton of variation across the industry. SDS stacks come in variants that focus on wildly different use cases:

• transactional use cases (think “vSAN”, think “ScaleIO”) – this will be the focus of this post.
• non-transactional file use cases (think “IsilonSD Edge”, “gluster”, etc.)
• non-transactional object use cases (think “Elastic Cloud Storage”, “CEPH”, “SWIFT”, etc.)
• all sorts of blends in between (including layering block, filesystem semantics on top of object stores)
• SDS “data services” – things that layer on value/data services that SDS’s (at least for now) have weak spots – the most obvious of these is DR (think “RecoverPoint”, “Zerto”, etc.)

The idea that there will be one “SDS way”, or that one of these stack approaches doesn’t hold up for second (though people who are super-passionate for one or the other sometimes hold that point of view – insane as it may be :-)

That said – is there an SDS pattern? Is there an SDS taxonomy for the transactional use case?

Like Part 1 – I won’t bury the lead. We believe that there are 2 fundamental SDS design points for transactional storage: those born for HCI ingredients, those born to be Server SANs.

• vSAN is our lead SDS whose central “design center” is the HCI world.
• ScaleIO is our lead SDS whose central “design center” is the Server SAN world.
• Currently we have #1, award winning, market leading offers in each category. THANKS customers!
• Yes, there is overlap, and I’m going to talk about it after the break, but like all nuanced topics, will require some focus and time investment from you, dear reader.

Don’t listen to me – I’m CLEARLY biased. Listen to Storage Review. They have completely pounded on storage stacks of all types over the years. BOTH vSAN and ScaleIO were Editor’s Choice winners – which you can see here for vSAN and here for ScaleIO.

So – what about overlap?   Well, for people who want a simple picture, here you go.

BIG green circle. BIG blue circle. Yes, they overlap, but they are pretty darn different – but the differences between the green circle and the blue circle are pretty vast. I’m going to explore the Venn diagram in this post – not only their differences, but yes, where they overlap (the purple).

Here’s an obvious hint: one is licensed by server; the other is licensed by TB – that’s a reflection of where they have their sweet spots.

Like Part 1 – this is not only your friendly neighborhood Virtual Geek’s point of view – it’s the official Dell Technologies point of view… so, if you want our answer in long winded, detailed, rambling Virtual Geek style – read on!

As I mentioned in Part 1 – we’ve spent a LOT of time collectively looking at this, all the way right to the top of VMware and Dell Technologies. We put the ideas to the test (because if you can cover the market with one stack, it’s ALWAYS better to do so).

So – why DO vSAN and ScaleIO both exist?

Both vSAN and ScaleIO are transactional SDS models. Both can cover cases where customers want to run VMs on an SDS. That statement is undeniable (and customers doing exactly that are legion).  Could we cover the transactional SDS market with one or the other?

Answer = nope, at least we don’t think so.  We think there are two distinct transactional SDS “design centers”.

1. SDS aimed squarely powering Hyper-Converged Infrastructure models. In this case, one almost never “sees” the storage layer – it’s completely embedded into the compute management construct – or simply invisible. The idea of a “storage management” interface is a strange one. The ideas of “pooling/sharing across many heterogenous use cases” is well – the opposite of the design center.
2. SDS aimed primarily as powering Server SAN models. In this case, the SDS is a proud storage thing first – broad use cases, in a broad set of topologies and heterogenous use cases. It can be deployed with compute running co-resident or 100% as a 2-layer SAN substitute…. But that all means it’s never that integrated with any single given use case, and clearly – it would need a rich “storage management” plane for use by someone who manages storage.

Don’t entirely see what I mean? Watch these two demos – and come right back.

Now that you’ve watched them – what did you observe?

First, let’s talk about the ScaleIO demo. In that demo, you see a very, very powerful SDS that can create a Server SAN that can replace many Enterprise SANs out there. ScaleIO has rich multi-tenancy (at the storage layer), QoS (at the storage layer) capability – and supports all sorts of heterogenous use cases. A SAN admin would evaluate it (and frankly in doing so, come out pretty impressed) as a SAN substitute. ScaleIO can happily replace the majority of SANs our there – supporting the majority of x86 workloads (vSphere and non-vSphere) – with the cases which aren’t an SDS fit noted in the first blog post in this series.

Second, let’s talk about vSAN. In the vSAN demo, you see a very, very powerful SDS that is used in a Hyper-Converged management context where it “fades into the background” of vSphere. vSAN has no “storage management” layer – because the natural management plane is vSphere. It has rich multi-tenancy and QoS capabilities – but at the VM level, not at the storage layer. It doesn’t “replace” or “substitute” for a SAN – it’s not a Server SAN. But - it’s more correct to say that it “eliminates the need for a shared storage thing of any kind” when it comes to vSphere scenarios. vSAN makes SDS for the the majority of x86 workloads for vSphere a feature of the hypervisor – with the cases which aren’t an SDS fit noted in the first blog post in this series.

Understanding this makes the answer to a series of questions really easy:

Let’s start by reinforcing the first blog post:

• Question: Should you be considering an SDS model? Answer: heck yeah. Software Defined Storage is ready for the majority of x86 workloads (by volume) as hyper-converged infrastructure or pooled storage. Continue to use SANs for specific workloads that demand very high capacity or performance, or have complex data protection needs.

Next, let’s ask the questions everyone is thinking about:

• Question: Why not use vSAN for all the workloads on vSphere that an SDS can support? Answer: Perhaps you could, but then you might miss out something. Some customers want an SDS that is a Server SAN – supporting vSphere and non-vSphere use cases, that doesn’t naturally follow the operational and technical boundaries of vSphere (whether it be cluster size, or average VM requirements).
• Question: Why not use ScaleIO for all workloads that an SDS can support? Answer: Perhaps you could, but then you might miss out something. Some customers want an SDS that is born to be an HCI SDS layer for vSphere – completely embedded, integrated, and in effect invisible.

Net: vSAN is an example of an SDS born to be an HCI ingredient and hyper-integrated with vSphere.

Net: ScaleIO is an example of an SDS born to be a general purpose Server SAN SDS, for vSphere and anything else.

This is codified in this picture below – the official Dell Technologies (VMware/Dell EMC) position:

Double clicking on the question of “what makes up SDS that powers HCI different than an SDS that powers Server SAN” is interesting.

When you ask the vSAN user base (a massive 7000+ customer base at this point and rising quickly – of every shape and size – from the small to the large, and all sorts of segments and workloads), they describe that they love how well vSAN works, but they really love how simple it is, how the policy is completely integrated, how it simply makes storage an extension of how they do software-defined compute. In this HCI operational model, you just want the storage to “be there” – the daily operational model is mostly driven by the VM, not by storage.

An important note – vSAN customers, should really update to 6.0u3 (more here, and a great post by the always great Cormac Hogan here). vSAN 6.0u3 has important performance improvements, and critical hardening improvements. I’ll do a standalone post on this, but VxRail customers, vSAN 6.0u3 is all rolled up in VxRail 4.0.131, which is a one-click update, and should be done by everyone. If you have an older version of VxRail (3.5) or use standard VMware licensing, call EMC support to help you (Dell EMC Technical Advisory 496692)

Conversely, when you talk to the ScaleIO user base (while smaller in count, tends to be deployed in larger deployments and customers – some of whom have massive scale, massive pooled capacity/performance requirements) they really love that their server SAN can be managed/operated/scaled in a way that is not directly linked to compute or network. It’s not uncommon that they are VERY large scale. It’s not that HCI models can’t scale big – but that like Hyper-Scale public clouds, more often than not – they deploy in models where SDS looks more like a Server SAN than an HCI.

ProTip to anyone thinking about portfolios, or product management: when you listen to your customers, you learn a lot :-)

As always dear reader, let’s speak frankly, directly, and transparently. Let’s put the elephant in the room on the table – do “HCI” and “Server SAN” overlap? Heck yes. Let’s talk about the “purple part” of the Venn diagram.

• First question: when does a Server SAN show up in an HCI form? Answer: VxRack FLEX. Why? Because VxRack FLEX is FLEXIBLE. It’s right in the name :-) Some customers want/need/demand an HCI operational model that can support vSphere well, but can support KVM, or Hyper-V, or bare-metal/physical use cases, or the container ecosystem (Docker/Mesos/Kubernetes) on Linux on bare-metal well. Recall, the operational model of HCI is about operating server/network/compute together. Sometimes they have needs that need a Server SAN – even in the 100% vSphere use cases. Like what? Like when they need to pool storage across vSphere clusters. Like when they need to have SDS storage scaling points that don’t fall into the majority of vSphere use cases that vSAN covers well.

The operational word for VxRack FLEX = FLEXIBLE. It also means that I intentionally used the word “well” when describing the support for a very broad set of use cases. Think about this – “flexibility” and “deeply integrated” are two design points that pull apart from each other.

While we are working to make the operational model and management/orchestration stack of VxRack FLEX better – it will NEVER be as simple/integrated for vSphere as VxRail or VxRack SDDC – but will always be more flexible.

Right now, and through 2017, VxRack SDDC is in directed availability. This means that while it is available and being deployed (not a “beta”), we are engaging a small (think 40-60) customers with both VMware and Dell EMC field teams. Why? We want to carefully handling every customer to strengthen muscles and have the product teams very directly engaged with the customers.

In 2018 we will light up VxRack SDDC for anyone who is a fit. Installs so far have been going really well, and we’re learning. Those learnings are immediately being fed back into the product – on the hardware and software parts of the stack. Even as we expand the integration with VxRack SDDC over 2017 for things ScaleIO life-cycle management out of the box, and for attaching external storage (important features in VxRack SDDC 2.0)… one thing will always be central to VxRack SDDC (and VxRail): operational model of VxRack SDDC is unabashedly focused on vSphere and the VMware SDDC stack. Deep integration is the design point.

Conversely VxRack FLEX is NOT unabashedly focused on vSphere and the VMware SDDC stack. Flexibility is the design point. VxRack FLEX is generally available and supports vSphere (perfectly fine, but not as tightly as VxRail and VxRack SDDC), but also is FLEXIBLE to support other use cases.

• Second question: when does an HCI SDS how up in Server SAN form? Answer: we’re starting to see cases where the SDS embedded in HCI is positioned for blended use cases and “storage only nodes”. We’ll see how that works – but I’m skeptical. An example of of the blended scenarios are where Nutanix can present to external hosts, and support storage-only nodes. Personally - I suspect (time will tell) that people will tend to bias to either HCI operational models, or Server SAN models (and in some customers – some of both). The design centers are pretty darn diametrically opposed (see Venn Diagram), and I can think of customers that are telling us they want these two distinct operational models.
• Third question: Why not just merge vSAN/ScaleIO? Put another way – why not just embed ScaleIO completely in the vSphere development path? Or Put another way – why not port vSAN to Linux and build a non-vCenter management plane? Then there’s no Venn diagram, no purple. People grapple with this because clearly both are technically possible. The answer is simple. Answer: because that would be a waste – and would take the engineering teams away from their sweet spots. Why would we take the best vSphere SDS whose design center is HCI (vSAN) and waste a great engineering team and R&D dollars/time to support use cases that ScaleIO covers well? Why would we take the best SDS whose design center that is Server SAN (ScaleIO) and waste a great engineering team and R&D dollars/time to make it cover use cases that vSAN covers well?

Look – the overlap does occur, but we’ve realized that it’s just silly to auger in on it.

• Sometimes the best answer for a customer is: “use vSAN for vSphere, ScaleIO for everything else”.
• Sometimes the best answer for a customer is: “use vSAN for vSphere – and that’s all you need”
• Sometimes the best answer for a customer answer is: “use just ScaleIO for vSphere and everything else”.
• Sometimes the best answer for a customer is: “even in a 100% vSphere use case them both – vSAN for the bulk of your vSphere use cases, and ScaleIO for some particular VMs”

And in the above, you can substitute the “appliance and rack-scale system consumption models” (VxRail, VxRack SDDC and VxRack FLEX) as appropriate.

How are things going in SDS and HCI land? Clearly this is important to us (read blog post part 1), and we don’t think that SDS and HCI are a passing fad.

While I’m always paranoid about drinking our own koolaid, an antidote to koolaid is market data and customer feedback.

We’re doing great, and customers really REALLY dig both vSAN and ScaleIO. Putting a fine point on it – Dell Technologies have the #1 HCI and #1 Server SAN SDS stacks.

In Q4, vSAN scaled past 7000 customers. In Q4, the VxRail numbers were enormous. In Q4, the ScaleIO deployments in all forms were massive – with multiple customers crossing the 20 Petabyte barrier – WOW.

With our self-disruptive “lean in” on SDS models of both types”, we want to be clear – we think this space is VERY important.

Remember the obvious hint: one is licensed by server; the other is licensed by TB – that’s a reflection of where they have the SDS sweet spots.

In the waning hours of 2016, back at the ranch at Dell EMC and VMware HQ, we realized that the “or” question was silly. We’re leading the market with vSAN and ScaleIO. Why not simply press that great advantage?

The best move for our customers, for ourselves was to spend no effort in one versus the other – but rather to as quickly as possible help customers discover the SDS approach that was the best for them. To make sure we have total, complete alignment and ensuring both the DellEMC and VMware field teams speak as one:

1. The position of vSAN as our lead SDS for HCI, and ScaleIO as our lead SDS for Server SAN – as well as the overlap around cases where ScaleIO powers VxRack FLEX - is universal across Dell Technologies.
2. We have operationalized it into both companies. Starting now in 2017 there is no role inside Dell EMC or VMware that has any compensation metric that will steer a customer to one side based on compensation or some “unnatural” driver. Sure, you might find people with passions that are particularly strongly one way or another (we are human after all, and tend to have areas of focus) – but vSAN and ScaleIO are both Dell Technologies SDS options for our customers – period.
3. The positions in this post series are not just my point of view and for Virtual Geek readers – but the authoritative position of Dell Technologies (both VMware and Dell EMC), and have been rolled out at the VMware WW kick-off, and will at the Dell EMC Field Readiness Session at the end of February. No one can “miss the memo” :-)

Like in Part 1, I would LOVE to hear your voice – both in comments and in a survey.

Please – share YOUR point of view – when it comes to transactional SDS models are you an HCI type, or a Server SAN type… or Both?    The next survey is here – 3 simple questions, less than 1 minute!   Like the last post, I will share all results openly and transparently, and of course, as always, comments welcome!