VMware Virtual Storage to Hook Up Flash with Docker

Version 6.6 of VMware’s vSAN component (formerly Virtual SAN), to be made available May 5, will include a Docker volume driver that will enable containerized applications and microservices to address virtual storage volumes as though they were persistent data containers.  It’s a move that VMware hopes should ease the transition of many enterprise applications from first-generation virtual machines to more adaptable, scalable components — while at the same time connecting containerized apps to volumes backed by the latest flash hardware.

“The new Docker volume driver gives you the ability to enable folks running native Docker to run vSAN on the back end with the native Docker APIs,” said Michael Haag, VMware’s group manager for product marketing, in a conversation with Data Center Knowledge.  “And we’re expanding the functionality to include our policy-based management capabilities, cloning, snapshots, and some other capabilities.”

Volume Driver

When the Docker wave was first catching on in 2014, there was a groundswell of excitement over the possibilities for data centers to distribute individual functions of running, cloud- and Web-based applications on demand.  The first developers to get excited, however, ran smack into an architectural hurdle that felt at the time like a sudden, stone wall:  There was no architecture yet for multiple copies of the same function to securely access the same database.

Proponents of a kind of “pure” containerization asserted that this roadblock was actually a guidepost toward a new kind of application development called the 12-factor methodology.  One of its tenets is described as statelessness, but which translates into a kind of isolationism: the idea that functions can scale up and down much easier if they share no data between themselves.

That kind of application architecture might just work if your first name is “Net-” and your last name “-flix.”  Out in the big world, however (breaking news, everyone), databases exist.  Existing applications use existing databases.  And you can’t migrate existing apps to new apps without carrying the data over with them.

“Docker does a great job of managing applications at a high level, splitting things apart, and making them more consumable,” explained Haag.  “But they haven’t dug into the storage side, where if something happens and your container goes away, how do you make sure that storage lives, and you don’t have to rebuild and re-create it?”

VMware’s Docker volume driver premiered in June 2016, but as a beta project hosted by the GitHub open source platform.  It has been one of the most ambitious applications of Docker’s platform extensibility, introduced the year before.  Its goal has been to enable vSphere administrators to create persistent storage volumes — components that don’t get instantiated and revoked like containers — but that remain under the administrative control of vSphere.

Of course, this plan has the virtue of leaving vSphere in the picture, for data centers considering a full-on move to containerization.  And for those data centers that are more ambitious, vSAN 6.6 is being made a native part of Photon, VMware’s containerization environment for those weaning themselves off of vSphere.

Plan B for Hyperconvergence

One of the main problems many data centers are facing with respect to decisions over hyperconverged infrastructure, is how to converge both legacy and modern workloads onto one platform.  Extraordinarily, some have suggested compartmentalizing systems into workload-specific blocks, enabling those that don’t play as nicely with HCI to be managed under separate policies.  Others suggest that workloads designed to run in parallel — specifically, those that use a lot of data, or that may generate variable amounts of latency — should perhaps be excluded from HCI platforms.

You’re reading this accurately:  Experts are saying you should stratify your hyperconvergence for safety’s sake.

By contrast, VMware has asserted that it’s taking a more converged (to coin a phrase) approach, by plotting routes for multiple components to be continually integrated onto single virtual platforms.  Last month, the company opted to reduce the number of virtual switch technologies it supported to one.  While some may be arguing that this reduces choice, there are certain areas of the infrastructure where choice for choice’s sake may be frivolous.

On the other hand, there may be other areas where certain features of a data center’s existing hardware have already made those choices for it.  This is the curious situation regarding NSX, the company’s network virtualization layer.  Last year, the confusion about whether customers should run vSAN with NSX together led then-principal architect Rawlinson Riviera (now the CTO of Global Field) to write a blog post stating 1) vSAN and NSX are “absolutely and unequivocally” compatible, but 2) that doesn’t mean you can just plug them together and run vSAN traffic over a VxLAN overlay.

As of now, said VMware’s Haag, you can plug the two compatible things together.  But you should want to first.

“From an HCI standpoint,” he told us, “it really starts with vSphere and vSAN.  If customers want to evolve to what we call the ‘full software-defined data stack’ with full network virtualization, NSX can definitely be added on top of that.  One of the benefits we have is the flexibility to talk about multiple deployment options.”

On the opposite side of the scale from Haag’s “full stack” is VxRail, which incorporates a turnkey version of VMware’s HCI stack, and which is currently being made available through sister company Dell EMC.

'Native' Moves to Software

Then there’s the case of native data-at-rest encryption.  Historically, vSAN has hit a stone wall when attempting to interface with self-encrypting drives, even if they’re from parent company Dell.  For vSAN 6.6, VMware is adopting the tack of building a single cross-platform function around encrypted data, presenting this as the “native” layer — and, in so doing, enabling the presumption that any other encryption option is a one-off.

“What we are introducing is a native, or software-defined, data-at-rest encryption,” remarked Haag.  “This is encryption built into the hypervisor, into the software stack, allowing us to fully encrypt all of the data that resides on any persistent media.”

Other HCI vendors, he contended, achieve what they portray as security by leveraging all its functions on some unique feature of the hardware — one of which is self-encrypting drives.  Because these leverages are each unique, he concluded, they present compliance problems.  And as long as an enterprise can’t be fully compliant, it isn’t truly secure.

“So by doing this in software, we’re allowing folks to deploy encryption on any certified hardware platform of their choice.  If they have existing SSDs in their systems, they can upgrade and deploy them on day one.  If they want to start leveraging the latest technology — the new Intel Optane NVME SSDs — and run in an encrypted environment, we’ll be able to support those out of the gate as well.”

That gate officially opens May 5.  At that time, VMware will begin licensing vSAN 6.6 for $2,495 per CPU, or $50 per desktop for VDI-restricted workloads.