ScaleIO the best multi-platform transactional SDS?

From where I sit, the storage market transition is in full swing.   

The era (1990-2015) where the world of “persistence” was dominated by “external” storage (aka “arrays”) is shifting to one where the majority of the growth is in SDS stacks running on industry-standard x86 hardware (leveraging a lot of NAND).  

These SDS persistence models are delivered in 4 “packaging” forms: a) software-only; b) bundled with servers (think VSAN-Ready Nodes, or ScaleIO-Ready Nodes); c) in hyper-converged infrastructure (appliance, or rack-scale); d) as a service (think Virtustream Cloud Storage, AWS S3, the Azure blob store, etc).

While the VAST majority of the market remains externalized storage, and there remain places where external all-flash arrays continue to rule the roost – anyone denying that the inflection point is upon us has their head in the sand.

• The huge acceleration in the VSAN, VxRail businesses, the ramp in the VxRack SDDC business are representations of that trend for customer who are “all in” on vSphere, and want a hyper-integrated vSphere stack.
• The huge acceleration in the ScaleIO, VxRack FLEX business are representations of that trend for customers who view infrastructure and transactional storage needing to support multiple stacks in a heterogenous way – in other words as a “substitute” for a SAN.

I’ll say one thing that I know for sure – there will NOT be a single SDS stack “to rule them all” – for the same reason that people who have argued for “one storage architecture” for years have been consistently and totally wrong: the design requirements for different workloads and data types are totally orthogonal.

This is why I’ve always scratched my head at “VSAN vs. ScaleIO”.   It’s an “and” not an “or”.  If VSAN is a great choice for customers all-in on vSphere,  ScaleIO is VSAN’s heterogeneous (including, but not only vSphere).   I’ll go so far as to make a bold statement:

I think ScaleIO may very well be the best multi-platform transactional SDS on the market.   It performs better, scales better, is more efficient at scale than any other SDS that supports many OSes, platforms, hypervisors, container ecosystems.

Why?

Well, for starters, every time that Storage Review has a go at it ScaleIO melts their faces.   The most recent example (SQL Server OLTP workloads in an HCI vs a two-tier configuration) is here.   Their words, not mine (but my emphasis):

“In our first look at application performance with EMC's ScaleIO VxRack Node in HCI, we are still surprised by just how well it performs. When running the application on the same hardware as the storage resources, ScaleIO software has no negative hit to the workload itself. This is an impressive feat in and of itself, considering other HCI platforms almost always sap a large percentage of CPU resources to manage background processes (some as high as 30%). To that end, our HCI results surpassed those that we measured with ScaleIO in two-layer mode, dropping average latency from 12.5ms average down to 10.3ms. For customers demanding the highest performing hyper-converged storage solution for SQL Server, ScaleIO just upped the bar again by being the most efficient HCI solution we've ever tested.

Well, for seconds, I can’t think of a transactional SDS other than CEPH RBD with this sort of OpenStack ecosystem support and integration (Cinder drivers included in all distros, Mirantis Fuel/Canonical Juju integration, Puppet, and much much more)… And ScaleIO outperforms (and similarly in resource efficiency and use) CEPH RBD by a factors of 7x-10x.   Wowsers - if you’re deploying (or thinking of deploying) OpenStack using KVM, you would be crazy to not put ScaleIO through it’s paces and compare it to your other SDS choices.   Missing out on a 7x-10x improvement is analagous to punching yourself in the face.    Take me up on my challenge, and share what you find.   GO FOR IT – the bits are here.   (and Randy Bias – thanks for continually pushing!)

Well, I’ll throw in a third…  there is no transactional SDS that is more integrated with the container ecosystem.   If there are any, please, hold up your hand, and I will update the post.   There’s integration with Docker, Mesos, Kubernetes, early Photon Platform work.   There’s open persistence volume controllers.  There’s authentication tooling.   There’s wrappers to help with Puppet, Go, Python, Flocker, and more.   You can get it all (and the source code) here.   BTW – if you want geo-distributed, super-duper object store S3-compliant SDS stack, you can get ECS (which also snaps into Cloud Foundry via the service broker listed at the link also :-)

And finally, if you want ScaleIO delivered in the best turnkey HCI Rack-scale form, VxRack FLEX is cooking, and more and more customers are going this way every day.   People talk about “web scale”.   This isn’t chest-beating, it’s about bringing the lower CAPEX (because you can start smaller, and don’t need to over-plan) of SDS and HCI, but even more about the operational benefits that SDS/HCI simplicity brings to bear.

This is a 16-cabinet VxRack FLEX going into a customer.   Why would ANYONE bother building their own – we are making this boring.   My job (more accurately the job of 2500 people at EMC and soon to be 3500 at Dell EMC) is to make infrastructure invisible.

People often ask “are these ready for mission critical stacks?”  Answer is “hells yeah”.   A recent ScaleIO customer picked it (in VxRack form) to support their Oracle RAC cluster at the core of their business.  If you’re a frequent traveller, you probably will use this system several times a week.    Now, the SDS stacks don’t have all the rich classic enterprise data services (high end remote replication is one area as an example) – but in practice for most customers, the workloads that need these things are the subset, not the superset of workloads.   In the example above, the customer was using Dataguard for remote replication/DR – and just needed a super-scalable, super-simple, super-performant, super-simple transactional SDS.  ScaleIO and VxRack FLEX fit the bill.

What does all this mean for the industry?

• If you’re a storage vendor or startup without an SDS stack, or one with a weak one (you know if I’m talking about you) – you better move fast, otherwise, you’re going to be left in the dust.   We have a plethora of riches – VSAN for customers all in on vSphere, ScaleIO for customers who want a transactional SDS that scales like a mofo and is heterogenous, IsilonSD Edge for NAS, ECS for a geo-distributed S3-compliant object store.
• If you’re a storage vendor or startup without a crazy awesome x86 server supply chain – you better get one fast.   While it’s not unfair to say the “hardware is secondary”, in HCI, the hardware is more than 50% of the COGS – so if you cannot compete, and can’t spare/depot globally, can’t bring the latest NAND, NGNVM, and leverage Intel’s roadmap – you’re going to struggle over time.
• If you’re a customer and not evaluating SDS options – you’re missing out.   Don’t take my word for it.   They are easy to access – and you don’t need to talk to a sales person to give them a shot (at least with us – all software stacks are available).  You have consumption choices ranging from software-only to appliance/HCI form factors.   I think it’s hyperbole (for now) to talk about SDS/HCI displacing all external storage – but it definitely an increasingly important part of your kitbag.

I’ll close with my bold black-and-white statement and let it hang out there for comments, agreement, or dispute!  

I think ScaleIO may very well be the best multi-platform transactional SDS on the market.   It performs better, scales better, is more efficient at scale than any other SDS that supports many OSes, platforms, hypervisors, container ecosystems.

Have you given ScaleIO a shot?  For what?  What have you found (good, bad, ugly – share!)