50Ply Blog

Building Things

Automating Debian Installs With Preseed and Puppet

| Comments

Setting up a batch of new servers can be tedious. Updating and maintaining those systems over their lifetimes is also tedious. Here’s some DevOps magic to relieve some of that tedium.

The end state we’re going for is a new installation of Debian Wheezy with Puppet installed and ready to maintain and update the configuration as our needs evolve.

To follow this recipe to-the-letter you will need an already existing Debian system to build the custom installation media. You will also need a web server to host some post-install scripts. I also have a Debian apt cache on my network to speed up the installation of multiple machines.

Step 1: Automate the Installation

There are a number of ways to do this in Debian and the wonderful Debian Administrator’s Handbook is where you can read about all of them. The technique I chose is called preseeding. The standard Debian installer is designed so that all of the questions it asks you can be answered in advance in a special file called the preseed file. Here’s a preseed file I put together.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

d-i debian-installer/locale string en_US
d-i console-keymaps-at/keymap select us
d-i netcfg/choose_interface select auto

# Use a local mirror instead of hitting the internet.
# You should change this to an appropriate mirror for your
# setup
d-i mirror/country string manual
d-i mirror/http/hostname string my-debian-mirror.mydomain.com
d-i mirror/http/directory string /debian
d-i mirror/http/proxy string

# Set up a user for me. This password is encrypted using the following
# command line:
#
# printf "r00tme" | mkpasswd -s -m md5
#
# Since I've disabled root login, my user will be set up automatically
# with the ability to sudo.
d-i passwd/root-login boolean false
d-i passwd/user-fullname string Brian Taylor
d-i passwd/username string btaylor
d-i passwd/user-password-crypted password $1$/Gd5fGd7$QNvq.odwNPXLo/HRuzdkw.

d-i clock-setup/utc boolean true
d-i time/zone string US/Eastern

# I also run NTP on my network so I'd like the installer to sync with
# my NTP server instead of going to the internet
d-i clock-setup/ntp boolean true
d-i clock-setup/ntp-server string my-time-server.mydomain.com

# Make all of the default decisions for partitioning my one drive. You
# can configure any setup that the installer can produce here but it
# can get complicated in a hurry
d-i partman-auto/disk string /dev/sda
d-i partman-auto/method string regular

# Shut-up any warnings if there are existing LVM or RAID devices
d-i partman-lvm/device_remove_lvm boolean true
d-i partman-md/device_remove_md boolean true

# Pick the "everything in one partition" layout
d-i partman-auto/choose_recipe select atomic

# Say yes to all the standard "are you sure you want to delete your
# disk" warnings.
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true

# Make sure my local mirror will land in /etc/apt/sources.list when
# we're all done here.
d-i apt-setup/local0/repository string http://my-debian-mirror.mydomain.com/debian/ squeeze main restricted universe multiverse

# Just a basic installation. But make sure I have SSH, puppet, and curl.
tasksel tasksel/first multiselect standard
d-i pkgsel/include string openssh-server puppet curl
d-i pkgsel/upgrade select safe-upgrade

# Put GRUB on that disk as well. Install anyway even if there's already
# an MBR on the diskd-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true

# I like to opt out of this
d-i popularity-contest/participate boolean false

# Once the installation is done we'll set the system up for some firstboot
# magic.
d-i preseed/late_command string chroot /target sh -c "/usr/bin/curl -o /tmp/postinstall http://my-web-server.mydomain.com/postinstall && /bin/sh -x /tmp/postinstall"

Step 2. Automate First-Boot Tasks

The final line of our preseed file runs a script at the end of the installation process. There’s not much room here so we just tell the installer to grab another script from our web-server and run that chrooted in our newly installed system. But, we’re still running under the installer and the installer’s post-install environment is pretty limiting. So, from our second script, we set up a service that will run a third script when the computer boots into its new installation for the first time. I’ll do all my real work in the third script.

Here’s the second postinstall script (the one that the script in the installer grabs and runs.) I host it at http://my-web-server.mydomain.com/postinstall:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

#!/bin/sh

# grab our firstboot script
/usr/bin/curl -o /root/firstboot http://my-web-server.mydomain.com/firstboot
chmod +x /root/firstboot

# create a service that will run our firstboot script
cat > /etc/init.d/firstboot <<EOF
### BEGIN INIT INFO
# Provides:        firstboot
# Required-Start:  $networking
# Required-Stop:   $networking
# Default-Start:   2 3 4 5
# Default-Stop:    0 1 6
# Short-Description: A script that runs once
# Description: A script that runs once
### END INIT INFO

cd /root ; /usr/bin/nohup sh -x /root/firstboot &


EOF

# install the firstboot service
chmod +x /etc/init.d/firstboot
update-rc.d firstboot defaults

echo "finished postinst"

Step 3. Upgrade to Wheezy

What should we do with our newly installed Debian Squeeze system? Upgrade it to Wheezy, of course! In my firstboot script, I use the preseed mechanism again to answer the questions that dist-upgrade is going to ask me. I swap out my sources.list with one that points to Wheezy and we’re off.

I wait until after I’ve upgraded to Wheezy to install my Puppet configuration. There are some changes to the Debian maintainer supplied version of the configs when we go from Squeeze to Wheezy and I want to make sure that my changes win.

This is the third post install script. It’s run the first time our newly installed Debian system boots. I host this script at http://my-web-server.mydomain.com/firstboot

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

#!/bin/sh


# This script will run the first time the system boots. Even
# though we've told it to run after networking is enabled,
# I've observed inconsistent behavior if we start hitting the
# net immediately.
#
# Introducing a brief sleep makes things work right all the
# time.
sleep 30

# install our new sources
cat > /etc/apt/sources.list <<EOF
deb http://my-debian-mirror.mydomain.com/debian wheezy main
EOF

# update apt
/usr/bin/apt-get update

# install our preseed so libc doesn't whine
cat > /tmp/wheezy.preseed <<EOF
libc6 glibc/upgrade boolean true
libc6 glibc/restart-services string
libc6 libraries/restart-without-asking boolean true
EOF

/usr/bin/debconf-set-selections /tmp/wheezy.preseed

# do the dist-upgrade
/usr/bin/apt-get -y dist-upgrade

# configure puppet to look for the puppetmaster at a specific
# machine. I really don't like the default of always naming
# the puppet master "puppet". This gets around that.

cat > /etc/default/puppet <<EOF
# Defaults for puppet - sourced by /etc/init.d/puppet

# Start puppet on boot?
START=yes

# Startup options
DAEMON_OPTS=""
EOF

cat > /etc/puppet/puppet.conf <<EOF
[main]
logdir=/var/log/puppet
vardir=/var/lib/puppet
ssldir=/var/lib/puppet/ssl
rundir=/var/run/puppet
factpath=$vardir/lib/facter
templatedir=$confdir/templates
server=my-puppet-master.mydomain.com
EOF

# Remove our firstboot service so that it won't run again
update-rc.d firstboot remove

# Reboot into the new kernel
/sbin/reboot

Now when the system comes up for the second time it will connect to the appropriate puppetmaster and we can manage it from there using Puppet manifests.

Creating the Installation Media

To start a new machine down our path-to-glory we’ll need some kind of installation media. There are instructions online for starting a preseed installation using network booting. I opted to build a CD image instead. Thankfully, the debian tool simple-cdd makes this process quite painless.

Here’s my simple-cdd config:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

# simple-cdd.conf minimal configuration file

# Note: this is only an example, it is recommended to only add configuration
# values as you need them.

# Profiles to include on the CD
profiles="default"

# set default locale
locale="en_US"

# Mirror for security updates
#   Expects security updates to be in dists/DEBIAN_DIST/updates
security_mirror="http://security.debian.org/"

Here are the steps:

Install simple-cdd

1

sudo apt-get install simple-cdd

Place the preseed file where simple-cdd can find it

1
2
3

cd ~
mkdir profiles
cp preseed profiles/default.preseed

Run simple-cdd with the provided configuration

1

simple-cdd --conf ./simple-cdd.conf

If all goes well you will have a freshly baked ISO waiting for you in your images directory. This is a standard awesome Debian ISO that can be burned to a disk or dd’d over to a USB flash drive. This ISO is sadly not the awesome standard Debian ISO that can be burned or dd’d to a USB disk. At the moment, the image produced by simple-cdd can only be burned to a CDROM.

Edit: s/Sid/Wheezy/ per Chris’s comment.

Edit: ISO can’t be dd’d to USB disk.

Comments