randy/ganeti-migration.md

Migrating a Ganeti Instance to a New Cluster

Assumptions

• Existing cluster master is, e.g., vm0.sea.rg.net AKA gnt0.sea.rg.net
• New cluster master is, e.g., vm4.sea.rg.net AKA gnt1.sea.rg.net
• There is a node on each cluster which has enough room in /var/lib/ganeti/exports for the largest image

On the target node, vm4.sea.rg.net, make sure /var/lib/ganeti/export is big enough to hold the largest image. if not,

lvcreate -L 200G -n ganeti-exports ganeti
mkfs /dev/ganeti/ganeti-exports

hack /etc/fstab to mount it

/dev/ganeti/ganeti-exports      /var/lib/ganeti/export  ext4    auto    0      0

and mount it

mkdir -p /var/lib/ganeti/export
mount /var/lib/ganeti/export

Export the Instance and Copy it to the New Cluster Node

Warning, as some of these transfers and imports are large, it would be wise to run under screen.

On the exporting system, vm0.sea.rg.net, export to a node with space, vm3 in this case.

gnt-backup export -n vm3 dataviz.org

Copy the exported image to the target node in the new cluster and delete it from source cluster. On the backup target node on the old cluster (-n above)

cd /var/lib/ganeti/export
rsyr dataviz.org vm4.sea.rg.net:`pwd`
gnt-backup remove dataviz.org

Import the Instance on the New Cluster Node

On the new cluster's master node, vm4 in this case, import the image

gnt-backup import -n vm4 -t plain dataviz.org

If you get sick dren such as

Resolved hostname 'static.118.158.21.65.clients.your-server.de' does not look the same as given hostname 'dataviz.org'

put the domain name in /etc/hosts with an unused address local to the LAN. And please do not ask why gnt-backup import looks the instance name up in the DNS.

Start and Test the Migrated Image

See if the image starts, being aware that the old exporting image may still be running and cause an IP address conflict. So, on exporting/old cluster

gnt-instance shutdown dataviz.org

and on the importing/new cluster

gnt-instance start dataviz.org

If it starts happily, it is safe to delete the import image on the new cluster, vm4 in this example.

gnt-backup remove dataviz.org

Of the node comes up but is not pingable, likely the LAN interface has been renamed. So ip link to find the new name, edit /etc/network/interfaces, and /etc/init.d/networking restart.

A Horrifying Alternative Hack

I have not gotten this hack to work inter-cluster over WAN.

okui.psg.com was 1001GB, and the above process would take massive disk and massive time. So i tried

On the target node

lvcreate -L 1001G -n ganeti-okui ganeti

If the lvcreate gets

Aborting. Failed to wipe start of new LV.

the hack is to do lvcreate -Zn ... to not try to zero the created logical volume.

As this will be a live block image copy, stop the running image at the source node

gnt-instance shutdown okui.psg.com

And back on the source node

dd bs=10M status=progress if=/dev/ganeti/84c258cb-dc37-46e1-b81e-b3fad1115854.disk0_data | ssh vm4.sea dd of=/dev/ganeti/ganeti-okui

Then use this grotty script

cat > /root/do-adopt << EOF
#!/bin/sh

# makeVM node lvm ramGB nameFQDN

# From: https://nsrc.org/activities/agendas/en/cloud-virtualization/cloud-virt/en/ganeti/ex-ganeti-instance-vmdk.html

NODE=$1
LVM=$2
RAM=$3
NAME=$4

gnt-instance add \
     -t plain \
     -o noop \
     --disk 0:adopt=$LVM \
     -B maxmem=${RAM}G,minmem=$((${RAM}/2))G \
     -n $NODE \
     -H kvm:vnc_bind_address=127.0.0.1 \
     --no-install \
     --no-start \
     --no-ip-check \
     --no-name-check \
     ${NAME}
EOF     

To adopt the dded image

/root/do-adopt vm4 ganeti-valid 2 okui.psg.com

If the Instance has multiple LVs

Hans came up with this hack for an instance which has more than one LV

gnt-instance add -t plain -o noop --disk 0:adopt=test_one --disk 1:adopt=test_two -B maxmem-1G,minmem=1G -n vm4 -H kvm:vnc_bind_address=127.0.0.1 --no-install --no-start --no-ip-check --no-name-check test-two-disks

If you get

Sun Feb 19 19:13:43 2023 Could not update metadata for instance 'gogs.rg.net': Error while executing backend function: Failed to start metadata daemon

then try

sudo gnt-cluster command chown gnt-metad:gnt-metad /var/log/ganeti/meta-daemon.log

If an instance was hiding on a hidden VLAN

 gnt-instance modify --net 0:modify,link=br-vlan42 gogs.rg.net

And Finally, ReName the New Cluster as gnt0

On the master node

gnt-cluster rename gnt0.sea.rg.net

---

2023.07.11