randy/migrating-ganeti-proxmox.md

Migrating a KVM Instance from Ganeti to Proxmox

We want to move a VM from a Ganeti node to a proxmox node.

I am sure all or most of this can be done through the Proxmox GUI, but I just did not want to figure that out. Others are welcome to try.

Initial Environment

The source Ganeti node, vm0.dfx.rg.net, has its VM instance storage using LVM group ganeti.

The target Proxmox node, vm2.dfw.rg.net, has its VM instance storage in a ZFS pool mounted as /images

We are going to move the instance named cache0.rg.net.

Some Prep to Make Things Simpler Later

So the console of the migrated VM does not do everything twice, make sure /etc/default/grub does NOT have console pramaters. It should only be

GRUB_CMDLINE_LINUX="console=tty0 console=ttyS0"

And remember to update-grub if you change it.

The interface name is likely to hange from ens13 to ens18. If you hack it in /etc/network/interfaces before migration you will not need to clean it up via the gunky ProxMox console access.

Moving the Disk Image

On the Ganeti node, shut down the instance.

sudo gnt-instance shutdown cache0.rg.net

If needed, and it usually is, make it plain, not drbd.

sudo gnt-instance modify -t plain --no-wait-for-sync cache0.rg.net

Find the filename of the image's disk. From gnt-instance info cache0.rg.net we see

  Disks: 
    - disk/0: plain, size 32.0G
      access mode: rw
      logical_id: ganeti/6a4c085f-b928-4304-91de-12a9145a3580.disk0_data
      on primary: /dev/ganeti/6a4c085f-b928-4304-91de-12a9145a3580.disk0_data (253:24)

To do the copy, pick a target filename to be obvious. And it is wise to run large copies under screen, especially if your keyboard machine is running MacOS Sequoia's broken ssh.

sudo dd bs=10M status=progress \
     if=/dev/ganeti/6a4c085f-b928-4304-91de-12a9145a3580.disk0_data \
     | ssh vm2.dfw.rg.net sudo dd of=/images/cache0

Note that using rsync occasionaly results in an image with grub failures when booting the target. But one would prefer an rsync recipe.

sudo -Es
rsync -vlpPtgoHx --copy-devices --copy-links \
     /dev/ganeti/6a4c085f-b928-4304-91de-12a9145a3580.disk0_data \
     vm2.dfw.rg.net:/images/cache0

Perhaps do a sha256sum or whatever of both files

vm0.dfx.rg.net:/home/randy> sudo sha256sum /dev/ganeti/6a4c085f-b928-4304-91de-12a9145a3580.disk0_data
bac18ccbedc9d8731846414d2cc52c58c8a17945f4635733f6d6a5691c9b5d74  /dev/ganeti/6a4c085f-b928-4304-91de-12a9145a3580.disk0_data
vm2.dfw.rg.net:/home/randy> sha256sum /images/cache0
bac18ccbedc9d8731846414d2cc52c58c8a17945f4635733f6d6a5691c9b5d74  /images/cache0

Create the VM on the Proxmox Node

We noted from the gnt-instance info above that the image's memory was 2048MB. Disk image sizing will automatically match the size of the image we just copied over.

We will use the Proxmox instance ID 100. You will want to change the vmid, memory, and name.

Create the image.

sudo qm create 100 \
    --autostart=1 \
    --onboot=1 \
    --boot order=virtio0 \
    --memory 2048 \
    --name cache0.rg.net \
    --serial0 socket \
    --net0 virtio,bridge=br-lan,firewall=1

Yes, autostart is currently ignored by Proxmox.

For some reason, perhaps a recent ProxMox update, causes UEFI to be the default boot. I can not see how to cofigure legacy on the qm create. So, to correct this, I go to the GUI, select the VM, choose Hardware, click BIOS, and use the pull-down.

Import the Migrated Image

Now import the disk image we copied over into the newly created VM.

sudo qm disk import 100 /images/cache0 images --format raw

Be sure disk and net are correct. Note the image name needs to match the vmid.

sudo qm set 100 --virtio0 file=images:vm-100-disk-0

Running the VM

We should now be able to start the VM

sudo qm start 100

It is easier to watch on an ASCII console where one can copy and paste

sudo qm terminal 100

For unknown reasons, the VM's Ethernet device name seems to change in the migration; commonly from ens13 to ens18. So we kinda like the console for this.

Cleaning up

You probably want to delete the copied image from the ZFS pool

sudo rm /images/cache0

You may also wish to ensure the old copy on ganeti does not come up again and surprise you

sudo gnt-instance modify --offline cache0.rg.net

If the VM is Multi-Disk

Copy both disks over, say to cache00 and cache01.

Do the import of cache00 normally.

remove the /images/foo file you just imported or it will be adopted on the rescan

on the target system

mkdir /var/lib/vz/images/<vmid>
cd /var/lib/vz/images/<vmid>
ln -s /images/cache01 cache01.raw
qm rescan --vmid <vmid>

You should now see it in the web GUI, and can Add it.

Be sure you add it to the images storage pool, not local.

Looking Around

We can see all images with

sudo qm list
      VMID NAME                 STATUS     MEM(MB)    BOOTDISK(GB) PID       
       100 cache0.rg.net        running    2048              32.00 6630

We will not see our VM(s) in the /images ZFS pool using classic ls /images. We need to use the Proxmox VE Storage Manager command.

sudo pvesm list images
Volid                Format  Type              Size VMID
images:vm-100-disk-0 raw     images     34359738368 100

And, if we want to see that VM image as a file

sudo pvesm path images:vm-100-disk-0
/dev/zvol/images/vm-100-disk-0

Warning: Do not ls -l /dev/zvol/images. It's a rabbit hole. :)

---

2026.08.24