randy/randy
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
randy/debian9-on-ganeti.md
Randy Bush b713113464
no recommends
2022-01-07 17:59:40 -08:00

8.5 KiB
Raw Permalink Blame History

Installing a Debian 9 Guest on an Ganeti Cluster

For this example, I will use the vm/host name vm2.iad.rg.net on the Dallas cluster. Modify to suit your needs.

Before I start the install, I put the new host in the DNS so that it has time to propagate.

Create the VM and do the Debian install

I cheat and use a script.

vm0.sjc.arrcus.com:/root# ./do-add vm1 128 32 sf.rg.net
Tue Jun 19 22:10:05 2018 * disk 0, size 128.0G
Tue Jun 19 22:10:05 2018 * creating instance disks...
Tue Jun 19 22:10:06 2018 adding instance sf.rg.net to cluster config
Tue Jun 19 22:10:06 2018 adding disks to cluster config
Tue Jun 19 22:10:06 2018 * wiping instance disks...
Tue Jun 19 22:10:06 2018  - INFO: * Wiping disk 0
Tue Jun 19 22:10:10 2018  - INFO:  - done: 0.8% ETA: 7m 40s
...
Tue Jun 19 22:17:25 2018  - INFO:  - done: 93.8% ETA: 29s
Tue Jun 19 22:17:55 2018  - INFO: Waiting for instance sf.rg.net to sync disks
Tue Jun 19 22:17:55 2018  - INFO: Instance sf.rg.net's disks are in sync
Tue Jun 19 22:17:55 2018  - INFO: Waiting for instance sf.rg.net to sync disks
Tue Jun 19 22:17:56 2018  - INFO: Instance sf
.rg.net's disks are in sync

Find the VNC console port and host of the new VM

gnt-instance list -o name,pnode,network_port sf.rg.net

Remember those for VNC.

Spin up the VM but boot from the Debian install CD-ROM

gnt-instance start -H boot_order=cdrom,cdrom_image_path=/ISOs/debian-9.13.0-amd64-netinst.iso sf.rg.net

From your local host, go for the VNC concole over a nice safe ssh tunnel. Tunnel to vm1.sjc.arrcus.com because that is where the instance lives. The ssh tunnel means you are relatively safe doing passwords etc over the wire.

$ ssh -N -L 5900:127.0.0.1:11002 vm1.sjc.arrcus.com

On my local host, a Mac, I start a VNC client (I use Chicken of the VNC) to localhost, port 5900, display 0, with the password of vnc on the target ganeti cluster.

Do the Debian install over the VNC GUI. Do not use graphical install if you wsnt to use manual IP addressing. The options I choose are:

  • US English and US keyboard. This means I am not offered the UTC timezone option and have to set manually later. But it sets the /etc/apt/sources.list to the US servers.

  • Manually set the IP address, but some folk like DHCP. In deb9, if you want to set the IP address manually, you have to when it asks about dhcp.

  • Use the entire disk. This will make a very hard to expand system. Alternatively, instead of this step, use https://git.rg.net/randy/randy/src/master/expanded-lvm-install.md

  • OpenSSH server as well as the standard system utilities. Not the desktop stuff. No print server.

  • Install the GRUB boot loader so that you can make serial console work. You may have to specify /dev/vda.

Reboot the VM

The installation completes, and the darn thing reboots the CD-ROM. So I get brutal, and back on the VM node I

gnt-instance reboot --shutdown-timeout=3 sf.rg.net

This usually whacks my VNC session, so I have to restart the client.

Because I did not use DHCP but set the address manually, I know where the host is. I immediately ssh into the new system as the new user.

On the other hand, if the IP address was assigned by DHCP, you'll have to stay with the VNC session and set up networking as below.

First Configuration

The first thing is set the password to something nasty. And remember to also set the user password.

Customize VM

I immediately set root's password to something nasty so that I can get in over the serial console if things get wedged during operation.

If it does not run Emacs, it is not a computer. And install unbound so we have a local nameserver, and ntpd so we have a local chime. We will do detailed configuration of unbound and chrony later.

cat > /etc/apt/apt.conf.d/01norecommend << EOF
APT::Install-Recommends "0";
APT::Install-Suggests "0";
EOF

apt install emacs-nox unbound chrony rsync sudo screen debian-goodies bsd-mailx

Hack /etc/sudoers to make it passwordless for members of the sudo group.

%sudo   ALL=(ALL) NOPASSWD: SETENV: ALL

Add the initial user to /etc/group in group sudo

usermod -a -G sudo randy

I want to disable ssh passwording as soon as reasonable, so I install root's ssh credentials.

cd /root
mkdir .ssh
chmod 700 .ssh
cat > .ssh/authorized_keys << EOF
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAID9DzOVboz/9daQwQRPkxD/D4PZ4ZVNgMLqd+zc79Gl+ randy@ryuu.psg.com
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAww1WTCBezBK1snfUKi1LAwIIvQEQuhgV63toCMnTrYZfkp9/jyI1WVmKf4FIiodJWWeUtjCkt2E7t0oPZ7BxNbP8bn6HTso8qTexng8kM23UCgdzorp0ofatO0esdIdbuXd8qu9uJRlJn1TwVYeY7qY10eF60CbSZawuwSsM0iq0EbKTJQkHBtytd1zC7a7geGfieR4D3sHB9b33YhUgP9b1EO+CYtS8Qbb8xbMboD1eIB0UH0jcSUVZvb2DRtnsyIVQioW6zaAbIbFz7x+5zJBsFjkDvjR/80atOvNYJTyYZRD71D3kBu2peYup2XrsTPvrsKh5hJCVlJFCuFQaNQ== randy@psg.com
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC8yvE/UBwHdLVDk708/U81ZQdtDLF9Y7grMkVBfUoclF4BGFCcH1bIv6y0p1I/2PATSRnK/B5ndvWP/SrQ6y0DqnQugHKoFMR3idkzVagjjXOhcZkpitcp5hRCdjn/y/88QEiIKItEM66q48nW0OtCgcq7ENXQuYUVt0e6kh6UKRDfrPP3i2D7m8o3/W+xYt8QkYZ0iFanCfhxHAP7w3SOezmThu+p7V6yxVcy+G0WTvPHrs2JlgSaI+4SoJ2btPgqIrmKu1ILfyoAr9/RjSVHr74LijsdUKec7OXnKnV2v1VfMRglBXFgXVaWyX/adA2tMuh9nMk3Y3a3+xDqPIz6RnzQAupqUmPZ/1qbFIrVSNmhygWTC9jgemaNNC/B1nFG7em6m+fmxKjauuWoIdpybaofWiJElUDBBT4bHU+L8NF7zbNROt1GtaDnbL4W9/asqzPu+aJy7L6uJQHtkc/eoXsheajztFC9I5qxW/29ja+FcQsVoDJpe96XmBtJ7elOGXaq3YtDRURwrdu9fGUBDWqwP1IFECVkeVh/NrXbQRHtgp4v533D+0MubA+hDhVTsUk6G93WLyX3usUprU1cJjyhrVzZ9U23eCPOj8J18Ixw4sJFuNeUhSCaN6j+rmOyGor3PYYysKTp1VfLNAYdcc/AYC0jDy9mYjjfCDwKfQ== randy@ryuu.psg.com
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIP2qXA3uRFGv7Q0yolQtC5bcF7kVW94c/aOhf0xbU3i8 randy@kappa.local
EOF
chmod 600 .ssh/authorized_keys

Copy the ssh authority to the user account

rsync -vlpPStgoHxr .ssh ~randy
chown -R randy:randy ~randy

Disallow plaintext password logins by editing /etc/ssh/sshd_config as root

# Change to no to disable tunnelled clear text passwords
#PasswordAuthentication yes
PasswordAuthentication no
UsePAM no

Take the new sshd config

service ssh restart

I want my standard dot files, so I go to a recently customized Xenial system and rsync them over.

rsync -vlpPStgoHx .bashrc .emacs .exrc .forward .inputrc randy@sf.rg.net:
ssh randy@sf.rg.net chown -R randy:randy .

And a bit of decoration from the other system

rsync -vlpPStgoHx /usr/bin/xtitle root@sf.rg.net:/usr/bin/xtitle

Configure GRUB and the Network

Configure GRUB for

  • longer timeout
  • serial console
  • use ntp not kvm clock
  • make the base ether eth0 instead of ens5

Edit /etc/default/grub

GRUB_TIMEOUT=6
...
GRUB_CMDLINE_LINUX="console=tty0 console=ttyS0,9600n8 no-kvmclock"

...
GRUB_TERMINAL="console serial"
GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"

And remember to update GRUB

update-grub

Last Cosmetics Before Reboot

If you are inside the Arrcus San Jose lab, there is a hidden DNS service inside the firewall, which you will want unbound to know. So copy the file over and restart unbound

rsync mgt.sjc.arrcus.com:/etc/unbound/unbound.conf.d/* /etc/unbound/unbound.conf.d/
service unbound restart

Edit /etc/hostname to fill out the full FQDN.

Set the timezone. All my servers are in UTC

dpkg-reconfigure tzdata

I still have the VNC console open, so it would be a good time to reboot and watch for problems. I could tell the VM to reboot, or whack it from the Ganeti master. To ensure it is really clean, I do the latter. I do not force the timeout so I can ensure that APCI works.

gnt-instance reboot sf.rg.net

Use Ansible to Install and Configure a Bunch of Things

Now the barrage of ansible, which for me is based on my Mac laptop.

ansible-playbook -l sf.rg.net no-depends.yml unattended-upgrades.yml chrony.yml rsyslog.yml snmpd.yml postfix.yml unbound.yml

Clean Up Unbound

To tell systemd to keep its claws off the DNS, the following hacks seem to work as of Ubuntu 18.04 after apt-get install unbound:

sudo systemctl disable systemd-resolved.service
sudo service systemd-resolved stop

Then tell unbound to regenerate its remote control junk.

sudo rm /etc/unbound/unbound_control.* /etc/unbound/unbound_server.*
sudo unbound-control-setup

Then make sure that /etc/resolv.conf points to localhost.

nameserver 127.0.0.1
nameserver ::1

Add the VM to Infrastructure Monitoring Etc.

Create an entry in srv0.iad.rg.net:/etc/nagios4/conf.d and enable checks in the GUI

Add the device to snmp monitoring https://nms.arrcus.com/

2022.01.07