# Build a Debian/Ganeti System on vm0.bknix.co.th

With much help from [https://nsrc.org/workshops/2014/sanog23-virtualization/raw-attachment/wiki/Agenda/ex-ganeti-install.htm](https://nsrc.org/workshops/2014/sanog23-virtualization/raw-attachment/wiki/Agenda/ex-ganeti-install.htm).

And [And https://nsrc.org/workshops/2014/sanog23-virtualization/wiki/Agenda](https://nsrc.org/workshops/2014/sanog23-virtualization/wiki/Agenda) is also helpful.

------

Boot into MegaRaid BIOS and configure the four drives as one RAID5 for the 1RU and RAID6 for the 2RUs.

### Install Debian

- Boot Debian CD/ISO
- Choose Install
- Choose English, UK (so you can get UCT)
- Choose American English
- Name the host
- Choose root password
- Choose user name and password
- Partition
  - Choose Manual Partitioning
  - Select the drive
  - Create new empty partition table
  - Select Free Space
  - Create new partition, primary, 1GB, begining, bios
  - Done
  - Select Free Space
  - Create new partition, primary, 256MGB, begining, /boot, ext4, bootable
  - Done
  - Select Free Space again
  - Create a new partition
  - Accept whatever size is shown (the rest of the disk)
  - Primary, physical volume for LVM
  - Done
- Configure LVM
  - Configure LVM accepting write changes to disks
  - Create volume group
    - Volume group name: ganeti
    - Devices for the new volume group: select only the LVM partition 
  - Create Logical Volume: on ganeti, root, 16G
  - Create Logical Volume: on ganeti, swap, 16G
  - Create Logical Volume: on ganeti, var, 16G
  - Edit the Logical Volumes to be ext4 /, swap, and ext4 /var
  - Finish partitioning and write changes
- Finish partitioning and write changes to disk

- Be sure it will not boot CD-ROM, and Reboot from the installed system

## Finish Debian Installation

Clean up from CDROM sources

```
vi /etc/apt/sources.list
```

and delete the two CDROM entries at the top

Install homey things (it's not a computer without emacs:)

```
apt-get update
apt-get upgrade
apt-get install emacs23-nox
apt-get install rsync
apt-get install gcc
apt-get install bridge-utils vlan
apt-get install sudo
apt-get install unbound
usermod -G sudo -a randy
```

Fix hostname

```
echo vm0.bknix.co.th > /etc/hostname
hostname `cat /etc/hostname`
```

Fix /etc/unbound/unbound.conf

```
        access-control: 127.0.0.0/8 allow
        access-control: 203.159.68.0/22 allow
        access-control: 0.0.0.0/0 refuse
        access-control: ::1 allow
        access-control: ::ffff:127.0.0.1 allow
        access-control: 2001:deb::/48
        access-control: ::0/0 refuse
```

### Unattended Upgrades, syslog-NG, etc.

[Install Unattended Upgrading](http://www.howtoforge.com/how-to-configure-automatic-updates-on-debian-squeeze)

Install syslog-ng

```
apt-get install syslog-ng
```

Hack /etc/logrotate.d/syslog-ng

```
*** /etc/logrotate.d/syslog-ng~ 2013-03-20 17:30:26.000000000 +0000
--- /etc/logrotate.d/syslog-ng  2014-07-31 14:00:08.148813531 +0100
***************
*** 29,34 ****
--- 29,36 ----
        missingok
        notifempty
        compress
+       mailfirst
+       mail randy@psg.com
        delaycompress
        sharedscripts
        postrotate
```

Fix Exim

```
dpkg-reconfigure exim4-config
```

Use internet mail
 System name is vm0.bknix.co.th
 Only listen on local interfaces
 Relay mail for noone

## Debian Ganeti Specific Configuration

Edit /etc/hosts to have the real address of the host, e.g.

```
127.0.0.1       localhost
203.159.71.10   vm0.bknix.co.th    vm0
203.159.71.11   vm1.bknix.co.th    vm1
203.159.71.12   vm2.bknix.co.th    vm2
203.159.71.42   gnt0.bknix.co.th   gnt0
```

### Fix /etc/network/interfaces

Make eth0 hang off of whatever your bridge will be called

```
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# public LAN
auto eth0
iface eth0 inet manual

auto br-public
iface br-public inet manual
      dns_nameservers 127.0.0.1 203.159.70.11 203.159.70.12
      dns_search bknix.co.th rg.net
      bridge_ports eth0
      bridge_stp off
      bridge_fd 0
      bridge_maxwait 0

# internal LAN
auto eth1
iface eth1 inet manual

iface eth1.222 inet manual
        vlan_raw_device eth1

iface eth1.444 inet manual
        vlan_raw_device eth1

auto br-internal
iface br-internal inet static
      address 203.159.71.10
      netmask 255.255.255.0
      gateway 203.159.71.1
      bridge_ports eth1.222
      bridge_stp off
      bridge_fd 0
      bridge_maxwait 0

auto br-cluster
iface br-cluster inet static
      address 10.0.1.10
      netmask 255.255.255.0
      bridge_ports eth1.444
      bridge_stp off
      bridge_fd 0
      bridge_maxwait 0

#iface eth0 inet6 auto
```

Check /etc.resolv.comf

In theory, this looks like

```
               -------------+--------------
                            |
                          br-lan 
                            |         this host
                  +---------+---------+
                  |        eth0       |
                  |                   |
                  |eth0.255   eth0.101|
                  +--+-----------+----+
                     |           |
                   br-svc      br-rep
                     |           |
         VMs --------+           +------> to other ganeti hosts
```

## Install Ganeti

Set up to get Ganeti from backports on all three servers, vm0, vm1, and vm2

```
cat > /etc/apt/sources.list.d/wheezy-backports.list
deb http://cdn.debian.net/debian/ wheezy-backports main
```

And then install it on all three servers

```
apt-get update
apt-get install ganeti/wheezy-backports
```

Fix up drbd

```
echo "options drbd minor_count=128 usermode_helper=/bin/true" > /etc/modprobe.d/drbd.conf
rmmod drbd      # ignore any error
modprobe drbd
```

### Initialize and Build Ganeti Cluster

```
gnt-cluster init \
  --master-netdev=br-internal \
  --enabled-hypervisors=kvm \
  -H kvm:kernel_path="",initrd_path="" \
  --vg-name=ganeti \
  -N link=br-cluster \
  -s 10.0.1.10 \
  gnt0.bknix.co.th
```

If it barfs, try

```
gnt-cluster destroy --yes-do-it
```

then re-run gnt-cluster init ...

If you get into gnutls problems check apt-get upgrade and dist-upgrade

### Check That Cluster is Happy

Fix VNC passwording

```
echo 'fnillies' > /etc/ganeti/vnc-cluster-password
gnt-cluster modify -H kvm:vnc_password_file=/etc/ganeti/vnc-cluster-password
```

Test that the cluster was built happily

```
gnt-cluster verify
```

If you get

```
Fri Mar 14 05:20:55 2014   - ERROR: node deb64.psg.com: volume ganeti/root is unknown
Fri Mar 14 05:20:55 2014   - ERROR: node deb64.psg.com: volume ganeti/swap is unknown
Fri Mar 14 05:20:55 2014   - ERROR: node deb64.psg.com: volume ganeti/var is unknown
```

That's OK.  To make it go away

```
gnt-cluster modify --reserved-lvs=ganeti/root,ganeti/swap,ganeti/var
```

Now add vm1 and vm2 to the cluster by the following on vm0

```
gnt-node add -s 10.0.1.11 vm1.bknix.co.th
gnt-node add -s 10.0.1.12 vm2.bknix.co.th
```

which should result in

```
vm0.bknix.co.th:/root# gnt-node add -s 10.0.1.11 vm1.bknix.co.th
-- WARNING -- 
Performing this operation is going to replace the ssh daemon keypair
on the target machine (vm1.bknix.co.th) with the ones of the current one
and grant full intra-cluster ssh root access to/from it

The authenticity of host 'vm1.bknix.co.th (203.159.71.11)' can't be established.
ECDSA key fingerprint is 2f:48:91:f7:f2:dd:c4:95:da:30:d2:f4:92:6a:53:ec.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'vm1.bknix.co.th' (ECDSA) to the list of known hosts.
Restarting OpenBSD Secure Shell server: sshd.
Fri Aug 22 05:37:53 2014  - INFO: Node will be a master candidate
```