Updating netboot image with newer kernel

This text describes the necessary steps to update a Debian netboot image with a more recent kernel (e.g. a kernel from backports or a custom kernel). It is written to be generic as possible, in the hope that it's useful for all Debian users (There's a part below which is specific to the production environment of the Wikimedia Foundation, but it can be skipped/adapted to your deployment)

The specific example used here is the integration of the backport of the stretch kernel (Linux 4.9) into a netboot image of jessie (which is based on 3.16 which doesn't support many of the NICs in use today).

• Download the kernel you want to integrate. In this case we're using the Linux 4.9 kernel for jessie (source package name linux-4.9), available from http://security.debian.org/pool/updates/main/l/linux-4.9/ (Ofc you can also build a custom kernel and use that)

• Extract the deb of the kernel package you want to use:

ar x $KERNELDEB
tar xf data.tar.xz (or possibly .bz or .gz)

• Now download the netinst image you want to modify. For jessie this is the netboot.tar.gz from http://ftp.debian.org/debian/dists/jessie/main/installer-amd64/current/images/netboot/

• Unpack the existing initrd image you want to modify

mkdir netboot
sudo unmkinitramfs initrd.gz netboot

Copy the content of lib/modules/$KERNELVERSION from the unpacked kernel deb to netboot/lib/modules (the complete directory including the modules.order and modules.buildin files).

• Now rebuild the initrd:

cd netboot
find . 2>/dev/null | cpio --quiet -H newc -o | gzip > ../initrd-new.gz

It's very likely that the netboot image will also need to be updated with firmware, in particular for PXE boots:

For that, download one of the firmware bundles which are available per Debian release: http://cdimage.debian.org/cdimage/unofficial/non-free/firmware/$DISTRO/current/firmware.tar.gz I.e. for our revised jessie image we need to download http://cdimage.debian.org/cdimage/unofficial/non-free/firmware/stretch/current/firmware.tar.gz It's important to get the right firmware.tar.gz release, as some firmwares are only compatible with specific versions of the Linux kernel.

A Debian initramfs can consist of one or several gzipped CPIO archives, as such we need to convert the firmware tarball to a CPIO archive:

mkdir firmware
tar -C firmware -zxf firmware.tar.gz
pax -x sv4cpio -s'%firmware%/firmware%' -w firmware | gzip -c >firmware.cpio.gz

• Next we need to append this our previously generated initrd:

cat firmware.cpio.gz >> initrd-new.gz

Now that we have an updated initrd, but you'll also need the kernel itself from the kernel deb extracted above: It's in the boot/ directory of the data.tar.xz extracted above and named like vmlinuz-KERNELVERSION-amd64

You need to add this initrd and kernel image in your PXE boot environment.

WMF-specific instructions for this below:

The tftpboot environment is distributed via puppet/volatile. Log into puppetmaster1001 and enter the /var/lib/puppet/volatile/tftpboot/ directory. Make a copy of the existing installer environment and e.g. copy it to e.g. jessie49-installer.

Now enter the jessie49-installer/debian-installer/amd64 directory and replace linux with the vmlinuz-KERNELVERSION-amd64 file extracted from the kernel and replace the initrd.gz with the one we rebuilt.

Finally the new kernel must be configured for the PXE boot. Edit modules/install_server/files/dhcpd/linux-host-entries.ttyS1-115200 in puppet and add/configure the following lines for the host to be installed:

  option pxelinux.pathprefix "http://apt.wikimedia.org/tftpboot/stretch414-installer/";
  filename "lpxelinux.0";

Now force a puppet run on the install servers via Cumin:

sudo cumin A:installserver 'run-puppet-agent'

The next installation would now use the jessie installer with the 4.9 kernel.