Background/Problem:

My KVM host, after a recent upgrade (see posts below) cannot start with the kernel option iommu=on enabled. Technically it can, however the system will not start due to a driver/bug issue with an additional SATA card I have installed:
:$ lspci
…
02:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller (rev 11)

Disks simply do not register when using iommu. The Bugzilla report can be found here and more information can be had here. The references are old, so my hope is that it has been patched in the latest kernel images.

Furthermore, I need the latest kernel to use the Quad tuner PCI-E card I have:
:$ lspci
…
05:00.0 Multimedia video controller: Conexant Systems, Inc. CX23887/8 PCIe Broadcast Audio and Video Decoder with 3D Comb (rev 04)

The quad tuner needs kernel 4.8 to run. More information here. So fundamentally, I need to compile the latest stable kernel image to get the full use of my system and then pass the PCIE tuner card through to my Media VM.

The Process:

I recommend doing this I have done, inside a reasonably powered Virtual machine. I’ve gone back through things and corrected my instructions when I’ve run into problems. This process will generate a Debian package that you can install on any Debian based OS (such as Ubuntu).

Problems:

1. Not giving enough RAM, CPU and disk space to the VM to compile (at all) or in a timely manner.
   1. I’ve given my VM 4 cores, 4GB RAM and an ‘external’ hard drive of 30GB to use to compile the kernel.
2. Utilize all the cores. add the line: CONCURRENCY_LEVEL= 4 to /etc/kernel-pkg.conf to use all 4 cores when compiling (once the package is installed, see below)
3. Not having some essential packages installed that caused the process to stop. such as libssl-dev.

Using all the cores makes it go much faster!

Steps:

At the command prompt, install all the packages you need to compile the kernel:
:$ sudo apt-get install fakeroot kernel-package gcc build-essential libncurses5-dev qt5-default libssl-dev

Download, to a disk that has ~20GB free, the latest stable kernel version. At the time of writing this was 4.9.9. Extract it and cd into the directory
:$ wget https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.9.9.tar.xz
:$ tar -xf linux-4.9.9.tar.xz
:$ cd linux.-4.9.9

Assuming you’re running this in a desktop Linux environment, run make xconfig, alternatively, if you’re using a terminal server, make menuconfig will do.

The default settings should do in most instances.

Save and close the configuration. Make the build environment clean, then begin the compile process:
:$ make-kpkg clean
:$ fakeroot make-kpkg –initrd –revision=4.9.9.linux kernel_image kernel_headers

You can save time by compiling a kernel with only the hardware that you have installed. Do this by deselecting them in the xconfig/menuconfig. The downside is that if you add new hardware, you’ll need to recompile the kernel.

For an explanation on the above fakeroot command, please see this Debian manual page. You should now have a custom kernel image compiling.

Once it’s completed, cd to the upper directory, and install the kernel:
:$ cd ..
:$ sudo dpkg -i linux-image-4.9.9_4.9.9.linux_amd64.deb linux-headers-4.9.9.9_4.9.9.9.linux_amd64.deb
:$ sudo shutdown -hr now

After restarting the VM, you can check the currently running version of the kernel by typing at the command prompt:

Additional extra step:

Prove to yourself that you’ve created a usable package by spinning up a shiny new VM, sftp the debian package to it, then install and reboot.

Further reading:

https://www.cyberciti.biz/faq/debian-ubuntu-building-installing-a-custom-linux-kernel/