I've been looking for an easy way to migrate Xen domains between my two Xen hypervisors. I made an (unsuccessful) attempt at setting up an iSCSI target for shared storage between the two servers. However, they are in different datacenters, so this is less than ideal.
Both servers share a similar setup-- storage for each domain's disk is held on a local LVM group, vg0. For instance, the DomU xr11 would be stored on /dev/vg0/xr11-disk on onodera, one of my servers. Therefore, to use the
xl migrate command to perform a "live" migration, this storage would simply need to be mirrored over to the other server. I use "live" loosely, since you wouldn't really want to be writing to the disk on your DomU while doing this. In fact, it's probably a good idea to issue a
sync command on the guest, then use
lvchange -pr to mark the volume as read-only until the migration is complete. But meh, we can't really be bothered with that.
Anyway, this can also be used to create backups of any LV to another server, or to assist in moving part of a volume group to another server (rather than the entire VG). While looking for some official LVM lvdump tool or similar, I found many folks just use
ssh to move volumes around their machines.
This script expands upon that idea-- provide it with the path to your LV (such as /dev/vg0/xr11-disk), and the server you would like to migrate it to, and it will create the LV on the new server with the exact same size, then copy the contents across with
ssh. I also decided to use
gzip with a fast compression setting, since most disks are quite sparse, and there's no sense in wasting bandwidth sending a bunch of zeros or other unimportant data. As you can see from the screenshots below, I was able to migrate a 30GiB volume in about 4 minutes (actual filesystem utilization is about 2GiB).
Once the transfer completes, the script also runs an
fsck -fp against the newly-copied volume to ensure filesystem consistency (in case you decided to copy the filesystem while the DomU was running).
The core of the script:
dd if=$LVPATH bs=1M status=none | pv --size=$LVSIZE | gzip -2 | ssh $NEWHOST -- "gunzip | dd of=$LVPATH bs=1M status=none"
After the migration completes, if you're moving a disk volume for a Xen guest, you can then run
xl migrate to do a live migration of the DomU to its new home (memory and state information should be retained). The screenshot below shows me testing this out, and it worked without a problem!
View lvmigrate.sh on Bitbucket or grab the script via wget, as shown below.
Also, be sure to install
pv, as this is not included with most distros by default. The line below should install
pv on Debian-based and RedHat-based distros, as well as Arch.
apt-get -y install pv || yum -y install pv || pacman -S pv
Grab the script:
chmod +x lvmigrate.sh