# Build a Software RAID5 Based Debian/Ganeti System only on a non-UEFI System **NOTE: this will not work on a UEFI capable system. It will fail at grub installation.** Try [This recipie instead.](https://wiki.rg.net/wiki/DebianRAIDGaneti) ## Install Debian - Before install, it is wise to disable DHCP service so one can manuall set the system IP data - Boot Debian CD/ISO - Choose Install - Choose English, UK (so you can get UCT) - Choose American English - Name the host - Select the Ethernet interface to be used - Configure Ethernet IP Address, Netmask, Gateway, and DNS Service - Configure Hostname and Domain Name - Choose root password - Choose user name and password - Partition Disks - Choose Manual Partitioning - Select the first drive - Create a new empty partition table - Select each of the other Drives and Create a new empty partition table - Configure Software RAID - Write Changes and Configure RAID - Create MD Device - Select RAID5 - Number of Devices should be all the drives you have - Number of Spare Drives is 0 - Select all the drives and continue - Write the changes and configure RAID - Finish - Configure LVM - Configure LVM accepting write changes to disks - Create volume group - Volume group name: ganeti - Devices for the new volume group: select /dev/md0 - Keep partioning and write - Create Logical Volume: on ganeti, root, 16G - Create Logical Volume: on ganeti, swap, 16G - Create Logical Volume: on ganeti, var, 16G - Finish - Edit the Logical Volumes to be ext4 /, swap, and ext4 /var - Finish partitioning and write changes - Finish partitioning and write changes to disk - Debian will now install and you will do the normal mirror selection etc. - Only install SSH Server and Debian Utilities - Install GRUB - Be sure it will not boot CD-ROM, and Reboot from the installed system ## Finish Debian Installation Install homey things (it's not a computer without emacs:) ``` apt-get update apt-get upgrade apt-get install emacs23-nox rsync gcc bridge-utils vlan sudo unbound usermod -G sudo -a randy ``` Copy root's credentials and dot-files from a known system Fix `/etc/ssh/sshd_config` to forbid passwords ``` PermitRootLogin without-password ``` And restart the ssh daemon ``` service ssh restart ``` Fix hostname ``` echo vm3.dfw.rg.net > /etc/hostname hostname `cat /etc/hostname` ``` Fix /etc/unbound/unbound.conf ``` access-control: 127.0.0.0/8 allow access-control: 198.180.152.0/24 allow access-control: 0.0.0.0/0 refuse access-control: ::1 allow access-control: ::ffff:127.0.0.1 allow access-control: 2001:deb::/48 access-control: ::0/0 refuse ``` And restart unbound ``` service unbound restart ``` Clean up from CDROM sources ``` vi /etc/apt/sources.list ``` and delete the two CDROM entries at the top ### Make ISO FileSystem ``` lvcreate -n ISOs -L 128G ganeti mkdir /ISOs mkfs /dev/mapper/ganeti-ISOs ``` and then add it to /etc/fstab ``` /dev/mapper/ganeti/ISOs /ISOs ext4 defaults 0 2 ``` and then you can mount it ``` mount /ISOs ``` ### Unattended Upgrades, syslog-NG, etc. [Install Unattended Upgrading](https://wiki.rg.net/wiki/UbuntuAutoUpgrade) Install syslog-ng ``` apt-get install syslog-ng ``` Hack /etc/logrotate.d/syslog-ng ``` *** /etc/logrotate.d/syslog-ng~ 2013-03-20 17:30:26.000000000 +0000 --- /etc/logrotate.d/syslog-ng 2014-07-31 14:00:08.148813531 +0100 *************** *** 29,34 **** --- 29,36 ---- missingok notifempty compress + mailfirst + mail randy@psg.com delaycompress sharedscripts postrotate ``` Fix Exim ``` dpkg-reconfigure exim4-config ``` Use internet mail System name is vm0.bknix.co.th Only listen on local interfaces Relay mail for noone ## Debian Ganeti Specific Configuration Edit /etc/hosts to have the real address of the host, e.g. ``` 127.0.0.1 localhost 198.180.152.30 vm0.dfw.rg.net vm0 198.180.152.31 vm1.dfw.rg.net vm1 198.180.152.32 vm2.dfw.rg.net vm2 198.180.152.33 vm3.dfw.rg.net vm3 198.180.152.100 gnt0.dfw.rg.net gnt0 ``` ### Fix /etc/network/interfaces Make eth0 hang off of whatever your bridge will be called ``` # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet manual auto br-lan iface br-lan inet static address 198.180.152.30 netmask 255.255.255.192 gateway 198.180.152.1 dns-nameservers 127.0.0.1 198.180.152.11 147.28.0.35 dns-search psg.com rg.net rpki.net bridge_ports eth0 bridge_stp off bridge_fd 0 bridge_maxwait 0 iface br-lan inet6 static address 2001:418:3807::30 netmask 64 gateway 2001:418:3807::1 auto eth1 iface eth1 inet manual auto br-hack iface br-hack inet static address 10.0.0.100 netmask 255.255.255.0 bridge_ports eth1 bridge_stp off bridge_fd 0 bridge_maxwait 0 ``` Check /etc.resolv.comf In theory, this looks like ``` -------------+-------------- | br-lan | this host +---------+---------+ | | | eth0 | | | +--------+----------+ | br-hack | +------> to other ganeti hosts ``` ## Install Ganeti Set up to get Ganeti from backports on all three servers, vm0, vm1, and vm2 ``` cat >> /etc/apt/sources.list.d/wheezy-backports.list deb http://cdn.debian.net/debian/ wheezy-backports main ``` And then install it on all three servers ``` apt-get update apt-get install ganeti/wheezy-backports ``` Fix up drbd ``` echo "options drbd minor_count=128 usermode_helper=/bin/true" > /etc/modprobe.d/drbd.conf rmmod drbd # ignore any error modprobe drbd ``` ### Initialize and Build Ganeti Cluster Only on the master node of the cluster ``` gnt-cluster init \ --master-netdev=br-lan \ --enabled-hypervisors=kvm \ -H kvm:kernel_path="",initrd_path="" \ --vg-name=ganeti \ -N link=br-hack \ -s 10.0.0.103 \ gnt0.dfw.rg.net ``` If it barfs, try ``` gnt-cluster destroy --yes-do-it ``` then re-run gnt-cluster init ... If you get into gnutls problems check apt-get upgrade and dist-upgrade ### Check That Cluster is Happy Fix VNC passwording ``` echo 'salad-daze' > /etc/ganeti/vnc-cluster-password gnt-cluster modify -H kvm:vnc_password_file=/etc/ganeti/vnc-cluster-password ``` Test that the cluster was built happily ``` gnt-cluster verify ``` If you get ``` Fri Mar 14 05:20:55 2014 - ERROR: node deb64.psg.com: volume ganeti/root is unknown Fri Mar 14 05:20:55 2014 - ERROR: node deb64.psg.com: volume ganeti/swap is unknown Fri Mar 14 05:20:55 2014 - ERROR: node deb64.psg.com: volume ganeti/var is unknown ``` That's OK. To make it go away ``` gnt-cluster modify --reserved-lvs=ganeti/root,ganeti/swap,ganeti/var ``` ## Add Nodes to Cluster Now add vm1, vm2, and vm3 to the cluster by the following on vm0 ``` gnt-node add -s 10.0.0.101 vm1 gnt-node add -s 10.0.0.102 vm2 gnt-node add -s 10.0.0.103 vm3 ``` which should result in ``` vm0.dfw.rg.net:/root# gnt-node add -s 10.0.0.103 vm3 -- WARNING -- Performing this operation is going to replace the ssh daemon keypair on the target machine (vm3.dfw.rg.net) with the ones of the current one and grant full intra-cluster ssh root access to/from it The authenticity of host 'vm3.dfw.rg.net (198.180.152.33)' can't be established. ECDSA key fingerprint is 04:31:79:b1:32:3b:6e:60:33:65:67:0f:76:9a:5a:36. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'vm3.dfw.rg.net' (ECDSA) to the list of known hosts. Restarting OpenBSD Secure Shell server: sshd. Sat Mar 21 22:25:47 2015 - INFO: Node will be a master candidate ```