vmdksync helps you escape from VMware

Posted: Sun, 13 May 2012 | permalink | 6 Comments

When I wrote lvmsync late last year, I didn’t realise I was being typecast. Before too long, I realised that the logic that I’d implemented for lvmsync would also help me with a separate migration project I’d been dreading – getting the day job off VMware.

Back in the early days of virtualisation, management made the decision to run VMware, for all the usual reasons (“commercially supported!”, “industry standard!”, and so on). Unsurprisingly (to me, anyway) it didn’t take too long for management to realise that it wasn’t the best choice for us. When you’ve got umpty-billion dollars to spend on hardware, software, and support, VMware might be the right option (although Amazon doesn’t seem to think so). Anchor’s company culture, on the other hand, is build around “smart staff, simple systems” over “dumb staff, smart vendors”, because no vendor is ever going to care about our customers as much as we do. So VMware was never going to work for us.

Unfortunately, as happens all too often, once VMware was in place, there was very little motivation to get rid of it and move those customers onto the chosen replacement (that we were deploying all new customers on). I happen to think this is a terrible attitude in general – one that makes life so much harder in the long term. I believe strongly in retrofitting old systems to keep them up-to-date with the current state of the art, and keeping technical debt under control. But, I wasn’t running the show back when we stopped putting new customers on VMware, so the few VMware servers we had stayed around far longer than they should have.

Recently, though, bad things started to happen. The VMware servers were starting to fall apart. The Windows machine we had to keep around to use the VMware management console started crapping out, and when the choice was between doing unspeakable things to Windows, and just ditching VMware… well, it wasn’t much of a choice. The only remaining question was how to do the migration off VMware with the least amount of downtime to our customers.

I was really quite surprised that nobody out in Internet land appeared to have come up with a simple, robust tool to do this. Sure, some vendors had all-singing, all-dancing toolkits that cost ridiculous amounts of money, required you to install their agent on the machine involved, and promised the earth, but it all smelt of snakeoil and bullshit.

In true hacker style, then, I decided to write something myself. The model I came up with mirrored lvmsync’s quite closely – because that one worked, and it turned out to be surprisingly easy to implement once I managed to reverse-engineer the file format (VMware has a PDF spec of a bunch of it’s file formats, but whoever wrote it was enough of an evil genius to make it utterly incomprehensible to anyone who doesn’t already know the file format, whilst making perfect sense to anyone who already does).

The result: vmdksync. It is nothing but 80-odd lines of ruby whose sole purpose is to take a delta.vmdk file and write the changes that are stored in that file to a file or block device that is a copy of the flat.vmdk file that you can copy while the VM is still running (after you’ve made a snapshot, of course). It helped me provide a painless migration path away from VMware, and I’d be really pleased if it helped some other people do the same. Share and enjoy!


From: Stig Sandbeck Mathisen
2012-05-13 17:39

The problem you describe is “technical debt”. It is a real problem, but often invisible to non-technical staff (who often decides what time and resources should be spent on).

It can be managed; Eric Allman recently wrote a very good article about this at http://queue.acm.org/detail.cfm?id=2168798

From: Matt Palmer
2012-05-14 06:55

Hi Stig,

I’m quite familiar with the term “technical debt” – hence why I wrote “keeping technical debt under control” in my post. Thanks for passing on the link to that ACM article; it certainly appears to be thought-provoking.

From: Michael
2012-05-15 02:37

Have you looked at virt-v2v : http://rwmj.wordpress.com/2010/10/07/guest-post-converting-vmware-guests-to-libvirtkvm-guests/ ?

( not sure if it support debian, but I am pretty sure that could help you to escape the vmware trap ).

From: Patrik
2012-05-15 06:40

Hi Matt, great article! I’m just curious… what’s the replacement for your old VMware setup?

From: Matt Palmer
2012-05-15 08:34

Michael: I hadn’t see virt-v2v, but it really solves a different problem – and, on the whole, one that doesn’t need to be solved. There’s no support for doing any sort of live conversion, which is the key thing that needs solving – taking VMs down for several hours for a simple migration is a really poor customer experience.

Patrik: We use a pretty straightforward KVM/libvirt setup. Fancy cloud management tools don’t make sense for us, and just make things far more complex for no benefit.

From: Darren
2013-03-27 02:10

It really boggles the mind that Redhat has not created live migration tools for KVM by now. Is this deliberate? Are there patents in the way? “Just shut down your Exchange server for a day or two while you wait for this command line conversion tool without a progress bar…” Thank you for vmdksync - a great tool until Redhat decides to take KVM seriously.

Post a comment

All comments are held for moderation; markdown formatting accepted.

This is a honeypot form. Do not use this form unless you want to get your IP address blacklisted. Use the second form below for comments.
Name: (required)
E-mail: (required, not published)
Website: (optional)
Name: (required)
E-mail: (required, not published)
Website: (optional)