randy/randy
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
randy/xenial-on-ganeti.md
Randy Bush 5595a85d21
more on fde
2017-12-06 14:08:52 -08:00

8 KiB
Raw Permalink Blame History

Installing a Ubuntu 16.04 Xenial Guest on an RGnet Ganeti Cluster

For this example, I will use the vm/host name random.sea.rg.net on the Westin cluster. Modify to suit your needs.

Before I start the install, I put the new host in the DNS so that it has time to propagate.

Create the VM and do the Ubuntu Install

I cheat and use a script.

vm0.sea.rg.net:/root# ./do-add vm0 32 4 random.sea.rg.net
Sat Feb 11 23:54:11 2017 * disk 0, size 32.0G
Sat Feb 11 23:54:11 2017 * creating instance disks...
Sat Feb 11 23:54:13 2017 adding instance random.sea.rg.net to cluster config
Sat Feb 11 23:54:13 2017 adding disks to cluster config
Sat Feb 11 23:54:14 2017  - INFO: Waiting for instance random.sea.rg.net to sync disks
Sat Feb 11 23:54:14 2017  - INFO: Instance random.sea.rg.net's disks are in sync
Sat Feb 11 23:54:14 2017  - INFO: Waiting for instance random.sea.rg.net to sync disks
Sat Feb 11 23:54:14 2017  - INFO: Instance random.sea.rg.net's disks are in sync

Find the VNC console of the new VM

vm0.sea.rg.net:/root# gnt-instance info random.sea.rg.net | grep console
  console connection: vnc to 127.0.0.1:11038 (node vm0.sea.rg.net) (display 5138)
serial_console: default (True)

Remember that 11038 for VNC.

Spin up the VM but boot from the Ubuntu install CD-ROM

gnt-instance start -H boot_order=cdrom,cdrom_image_path=/ISOs/ubuntu-16.04.2-server-amd64.iso random.sea.rg.net

From your local host, go for the VNC concole over a nice safe ssh tunnel. Tunnel to vm0.sea.rg.net because that is where the instance lives.

$ ssh -N -L 5900:127.0.0.1:11038 vm0.sea.rg.net

On my local host, a Mac, I start a VNC client (I use Chicken of the VNC) to localhost, port 5900, display 0, with the password of vnc on the target ganeti node.

Do the Ubuntu install on the VNC GUI. The options I choose are:

  • US English and US keyboard. This means I am not offered the UTC timezone option and have to set manually later. But it sets the /etc/apt/sources.list to the US servers.

  • Manually set the IP address, but some folk like DHCP.

  • Use the entire disk with LVM. This will make a very hard to expand system. Alternatively, instead of this step, use https://git.rg.net/randy/randy/src/master/expanded-lvm-install.md

  • Automatic security updates, but I will hack this further later.

  • OpenSSH server as well as the standard system utilities

  • Install the GRUB boot loader so that I can make serial console work

Reboot the VM

The installation completes, and the darn thing reboots the CD-ROM. So I get brutal, and back on the VM node I

gnt-instance reboot --shutdown-timeout=3 random.sea.rg.net

This usually whacks my VNC session, so I have to restart the client.

Because I did not use DHCP but set the address manually, I know where the host is. I immediately ssh into the new system as the new user.

First Configuration

The first thing is set the password to something nasty.

Then I become root, and get the software current.

randy@random:~$ sudo -Es
root@random:~# apt update
root@random:~# apt upgrade

This can take a while.

Customize VM

If it does not run Emacs, it is not a computer. And install unbound so we have a local nameserver, and ntpd so we have a local chime. We will do detailed configuration of unbound and chrony later.

apt install emacs-nox unbound chrony
chown -R unbound:unbound /etc/unbound/

I set root's password to something nasty so that I can get in over the serial console if things get wedged during operation.

I want to disable ssh passwording as soon as reasonable, so I install root's ssh credentials. These are not real.

cd /root
mkdir .ssh
chmod 700 .ssh
cat > .ssh/authorized_keys
ssh-ed25519 AAAAC3NzaC1lZDI1NE5AAAAIDzOVboz/9daQwQRPkxD/TD4PZ4ZVNgMLqd+zc79Gl+ randy@ryuu.psg.com
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC8yvE/UBwHdLVDk708/U81ZQdtDLF9Y7grMkVBfUoclF4BGFCcH1bIv6y0p1I/2PATSRnK/B5ndvWP/SrQ6y0DqnQugHKoFMR3idkzVagjjXOhcZkpitcp5hRCdjn/y/88QEiIKItEM66q48nW0OtCgcq7ENXQuYUVt0e6kh6UKRDfrPP3i2D7m8o3/W+xYt8QkYZ0iFanCfhxHAP7w3SOezmThu+p7V6yxVc+G0WTvPHrs2JlgSyaI+4SoJ2btPgqIrmKu1ILfyoAr9/RjSVHr74LijsdUKec7OXnKnV2v1VfMRglBXFgXVaWyX/adA2tMuh9nMk3Y3a3+xDqPIz6RnzQAupqUmPZ/1qbFIrVSNmhygWTC9jgemaNNC/B1nFG7em6m+fmxoIdpybaofWiJElUDBBT4bHU+L8NF7zbNROt1GtaDnbL4W9/asqzPu+aJy7L6uJQHtkc/eoXsheajztFC9I5qxW/29ja+FcQsVoDJpe96XmBtJ7elOGXaq3YtDRURwrdu9fGUBDWqwP1IFECVkeVh/NrXbQRHtgp4v533D+0MubA+hDhVTsUk6G93WLyX3usUprU1cJjyhrVzZ9U23eCPOj8J18Ixw4sJFuNeUhSCaN6j+rmOyGor3PYYysKTp1VfLNAYdcc/AYC0jDy9mYjjfCDwKfQ== randy@ryuu.psg.com
chmod 600 .ssh/authorized_keys

Copy the ssh authority to the user account

rsync -vlpPStgoHxr .ssh ~randy
chown -R randy:randy ~randy

Disallow plaintext password logins by editing /etc/ssh/sshd_config as root

# Change to no to disable tunnelled clear text passwords
#PasswordAuthentication yes
PasswordAuthentication no

Take the new sshd config

service ssh restart

I want my standard dot files, so I go to a recently customized Xenial system and rsync them over.

rsync -vlpPStgoHx .bashrc .emacs .exrc .forward .inputrc randy@random.sea:
ssh randy@random.sea chown -R randy:randy .

And a bit of decoration from the other system

rsync -vlpPStgoHx /usr/bin/xtitle root@random.sea:/usr/bin/xtitle

Undo the root login hack to /etc/ssh/sshd_config

PermitRootLogin prohibit-password

Take the new sshd config

service ssh restart

Configure GRUB and the Network

Configure GRUB for

  • longer timeout
  • serial console
  • use ntp not kvm clock
  • make the base ether eth0 instead of ens5

Edit /etc/default/grub

GRUB_TIMEOUT=6
...
GRUB_CMDLINE_LINUX="net.ifnames=0 biosdevname=0 console=tty0 console=ttyS0,9600n8 no-kvmclock"

...
GRUB_TERMINAL="console serial"
GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"

And remember to update GRUB

update-grub

systemd can be really unhelpful with the console. The following incantation helps

cat > /lib/systemd/system/ttyS0.service << EOF
[Unit]
Description=Serial Console Service

[Service]
ExecStart=/sbin/getty -L 115200 ttyS0 vt102
Restart=always

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable ttyS0
systemctl start ttyS0

IMMEDIATELY udate /etc/network/interfaces to change the name of the ethernet to eth0 and any other tunings.

auto eth0
iface eth0 inet static
  address 147.28.0.2/24
  gateway 147.28.0.1
  dns-nameserver 127.0.0.1
  dns-nameserver 147.28.0.35
  dns-search rg.net psg.com

iface eth0 inet6 static
  address 2001:418:1::2/64
  gateway 2001:418:1::1

Last Cosmetics Before Reboot

Edit /etc/hostname to fill out the full FQDN.

Set the timezone. All my servers are in UTC

dpkg-reconfigure tzdata

Hack /etc/sudoers to make it passwordless for members of the sudo group.

%sudo   ALL=(ALL) NOPASSWD: SETENV: ALL

I still have the VNC console open, so it would be a good time to reboot and watch for problems. I could tell the VM to reboot, or whack it from the Ganeti master. To ensure it is really clean, I do the latter. I do not force the timeout so I can ensure that APCI works.

gnt-instance reboot random.sea.rg.net

Use Ansible to Install and Configure a Bunch of Things

First, to use Ansible, the host must have python2, and have it linked to be python

apt install python2.7
ln /usr/bin/python2.7 /usr/bin/python

Now the barrage of ansible, which for me is based on my Mac laptop.

ansible-playbook -l random.sea.rg.net no-depends.yml snmpd.yml syslog-ng.yml unattended-upgrades.yml unbound.yml chrony.yml

Add the VM to Infrastructure Monitoring Etc.

Create an entry in srv0.iad.rg.net:/etc/nagios3/conf.d and enable checks in the GUI

Add the device to snmp monitoring https://nms.rg.net/

And finally, use Exim as the mail transport and install a client.

apt install exim4 mailutils
dpkg-reconfigure exim4-config

Edit /etc/aliases to send root mail where you really want it to go, and run newaliases

2017.10.05