randy/rgnet-wiki
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
rgnet-wiki/pages/GanetiMD.md
Randy Bush 6c281e5a6c from wiki
2020-07-04 14:55:25 -07:00

8 KiB
Raw Blame History

Build a Software RAID5 Based Debian/Ganeti System only on a non-UEFI System

NOTE: this will not work on a UEFI capable system. It will fail at grub installation. Try This recipie instead.

Install Debian

  • Before install, it is wise to disable DHCP service so one can manuall set the system IP data
  • Boot Debian CD/ISO
  • Choose Install
  • Choose English, UK (so you can get UCT)
  • Choose American English
  • Name the host
  • Select the Ethernet interface to be used
  • Configure Ethernet IP Address, Netmask, Gateway, and DNS Service
  • Configure Hostname and Domain Name
  • Choose root password
  • Choose user name and password
  • Partition Disks
    • Choose Manual Partitioning
    • Select the first drive
    • Create a new empty partition table
    • Select each of the other Drives and Create a new empty partition table
    • Configure Software RAID
    • Write Changes and Configure RAID
    • Create MD Device
    • Select RAID5
    • Number of Devices should be all the drives you have
    • Number of Spare Drives is 0
    • Select all the drives and continue
    • Write the changes and configure RAID
    • Finish
  • Configure LVM
    • Configure LVM accepting write changes to disks
    • Create volume group
      • Volume group name: ganeti
      • Devices for the new volume group: select /dev/md0
      • Keep partioning and write
    • Create Logical Volume: on ganeti, root, 16G
    • Create Logical Volume: on ganeti, swap, 16G
    • Create Logical Volume: on ganeti, var, 16G
    • Finish
    • Edit the Logical Volumes to be ext4 /, swap, and ext4 /var
    • Finish partitioning and write changes
  • Finish partitioning and write changes to disk
  • Debian will now install and you will do the normal mirror selection etc.
  • Only install SSH Server and Debian Utilities
  • Install GRUB
  • Be sure it will not boot CD-ROM, and Reboot from the installed system

Finish Debian Installation

Install homey things (it's not a computer without emacs:)

apt-get update
apt-get upgrade
apt-get install emacs23-nox rsync gcc bridge-utils vlan sudo unbound
usermod -G sudo -a randy

Copy root's credentials and dot-files from a known system

Fix /etc/ssh/sshd_config to forbid passwords

PermitRootLogin without-password

And restart the ssh daemon

service ssh restart

Fix hostname

echo vm3.dfw.rg.net > /etc/hostname
hostname `cat /etc/hostname`

Fix /etc/unbound/unbound.conf

        access-control: 127.0.0.0/8 allow
        access-control: 198.180.152.0/24 allow
        access-control: 0.0.0.0/0 refuse
        access-control: ::1 allow
        access-control: ::ffff:127.0.0.1 allow
        access-control: 2001:deb::/48
        access-control: ::0/0 refuse

And restart unbound

service unbound restart

Clean up from CDROM sources

vi /etc/apt/sources.list

and delete the two CDROM entries at the top

Make ISO FileSystem

lvcreate -n ISOs -L 128G ganeti
mkdir /ISOs
mkfs /dev/mapper/ganeti-ISOs

and then add it to /etc/fstab

/dev/mapper/ganeti/ISOs /ISOs   ext4    defaults        0       2

and then you can mount it

mount /ISOs

Unattended Upgrades, syslog-NG, etc.

Install Unattended Upgrading

Install syslog-ng

apt-get install syslog-ng

Hack /etc/logrotate.d/syslog-ng

*** /etc/logrotate.d/syslog-ng~ 2013-03-20 17:30:26.000000000 +0000
--- /etc/logrotate.d/syslog-ng  2014-07-31 14:00:08.148813531 +0100
***************
*** 29,34 ****
--- 29,36 ----
        missingok
        notifempty
        compress
+       mailfirst
+       mail randy@psg.com
        delaycompress
        sharedscripts
        postrotate

Fix Exim

dpkg-reconfigure exim4-config

Use internet mail System name is vm0.bknix.co.th Only listen on local interfaces Relay mail for noone

Debian Ganeti Specific Configuration

Edit /etc/hosts to have the real address of the host, e.g.

127.0.0.1          localhost
198.180.152.30     vm0.dfw.rg.net vm0
198.180.152.31     vm1.dfw.rg.net vm1
198.180.152.32     vm2.dfw.rg.net vm2
198.180.152.33     vm3.dfw.rg.net vm3
198.180.152.100    gnt0.dfw.rg.net gnt0

Fix /etc/network/interfaces

Make eth0 hang off of whatever your bridge will be called

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet manual

auto br-lan
iface br-lan inet static
  address 198.180.152.30
  netmask 255.255.255.192
  gateway 198.180.152.1
  dns-nameservers 127.0.0.1 198.180.152.11 147.28.0.35
  dns-search psg.com rg.net rpki.net
  bridge_ports    eth0
  bridge_stp      off
  bridge_fd       0
  bridge_maxwait  0

iface br-lan inet6 static
  address 2001:418:3807::30
  netmask 64
  gateway 2001:418:3807::1

auto eth1
iface eth1 inet manual

auto br-hack
iface br-hack inet static
    address         10.0.0.100
    netmask         255.255.255.0
    bridge_ports    eth1
    bridge_stp      off
    bridge_fd       0
    bridge_maxwait  0

Check /etc.resolv.comf

In theory, this looks like

               -------------+--------------
                            |
                          br-lan 
                            |         this host
                  +---------+---------+
                  |                   |
                  |       eth0        |
                  |                   |
                  +--------+----------+
                           |
                        br-hack
                           |
                           +------> to other ganeti hosts

Install Ganeti

Set up to get Ganeti from backports on all three servers, vm0, vm1, and vm2

cat >> /etc/apt/sources.list.d/wheezy-backports.list
deb http://cdn.debian.net/debian/ wheezy-backports main

And then install it on all three servers

apt-get update
apt-get install ganeti/wheezy-backports

Fix up drbd

echo "options drbd minor_count=128 usermode_helper=/bin/true" > /etc/modprobe.d/drbd.conf
rmmod drbd      # ignore any error
modprobe drbd

Initialize and Build Ganeti Cluster

Only on the master node of the cluster

gnt-cluster init \
  --master-netdev=br-lan \
  --enabled-hypervisors=kvm \
  -H kvm:kernel_path="",initrd_path="" \
  --vg-name=ganeti \
  -N link=br-hack \
  -s 10.0.0.103 \
  gnt0.dfw.rg.net

If it barfs, try

gnt-cluster destroy --yes-do-it

then re-run gnt-cluster init ...

If you get into gnutls problems check apt-get upgrade and dist-upgrade

Check That Cluster is Happy

Fix VNC passwording

echo 'salad-daze' > /etc/ganeti/vnc-cluster-password
gnt-cluster modify -H kvm:vnc_password_file=/etc/ganeti/vnc-cluster-password

Test that the cluster was built happily

gnt-cluster verify

If you get

Fri Mar 14 05:20:55 2014   - ERROR: node deb64.psg.com: volume ganeti/root is unknown
Fri Mar 14 05:20:55 2014   - ERROR: node deb64.psg.com: volume ganeti/swap is unknown
Fri Mar 14 05:20:55 2014   - ERROR: node deb64.psg.com: volume ganeti/var is unknown

That's OK. To make it go away

gnt-cluster modify --reserved-lvs=ganeti/root,ganeti/swap,ganeti/var

Add Nodes to Cluster

Now add vm1, vm2, and vm3 to the cluster by the following on vm0

gnt-node add -s 10.0.0.101 vm1
gnt-node add -s 10.0.0.102 vm2
gnt-node add -s 10.0.0.103 vm3

which should result in

vm0.dfw.rg.net:/root# gnt-node add -s 10.0.0.103 vm3
-- WARNING -- 
Performing this operation is going to replace the ssh daemon keypair
on the target machine (vm3.dfw.rg.net) with the ones of the current one
and grant full intra-cluster ssh root access to/from it

The authenticity of host 'vm3.dfw.rg.net (198.180.152.33)' can't be established.
ECDSA key fingerprint is 04:31:79:b1:32:3b:6e:60:33:65:67:0f:76:9a:5a:36.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'vm3.dfw.rg.net' (ECDSA) to the list of known hosts.
Restarting OpenBSD Secure Shell server: sshd.
Sat Mar 21 22:25:47 2015  - INFO: Node will be a master candidate