Horribly convoluted Linux kernel build processes (for a distribution)

Suppose you want to build a new kernel RPM that incorporates a different kernel (slightly up or down from distribution baseline). You want to turn off all their patches, and simply build the kernel, the headers, the -devel, …

Can you do it?

No I am serious… can you do it? The following is a bit of a rant. Borne out of frustration with things that are designed broken (IMO).

Part of the problem is, not surprisingly, RPM itself. The other part of the problem comes from the spec file. In the simplest sense, a spec file details where things come from, and how to build them.

We have built a number of packages as RPMs, and it hasn’t ever been simple, even if building the package was as simple as “./configure ; make ; make install”.

This presents, well, at best case, a significant problem … no … call it a barrier … to delivering functionality. You cannot *easily* modify the kernel rpms, in order to incorporate new drivers, or downrev the kernel to support a driver.

Say for example, downreving the Fedora 9 kernel to 2.6.24 to be able to support Infiniband OFED stacks.

Nah, that wouldn’t be useful. Not at all.

So far the *only* sane distro kernel build system I have seen for a distro has been the ubuntu version. This is based upon Debian.

Outside of the distro build system, we can build linux kernels just fine. Its the package manager that gets in the way.

Go figure.

RPM has been a pain for us for years, and we have stuck with building with it. I think it is high time we rethought package management, specifically the implementation of RPM. As it is, it is IMO horribly broken. Just look at a recent kernel source rpm. This is not a thing of beauty or elegance. It is a cave filled with razor sharp edges and no lights, just waiting to open up a few blood vessels.

I’ll revisit the build and use our own SPEC file. A minimalistic spec file.

Maybe I can use alien to convert our Ubuntu .debs into .rpms without having to go through the horrible build process known as RPM.

Viewed 13352 times by 3096 viewers

2 thoughts on “Horribly convoluted Linux kernel build processes (for a distribution)

  1. What the Red Hat/Fedora/CentOS HPC community needs is a script/build system/something that will essentially allow a complete “make world”. The resultant RPMs can be used to ‘upgrade’ (or ‘downgrade’, as is the case in your example). This ‘system’ can be used when changes are made to the kernel, OFED (new versions or vendor tweaks), MPIs (same), math libraries, or whatever.

    It seems that most of the provisioning tools will layout a base system, but none that I am aware of will allow rebuilding themselves in a systematic, package system compliant way.

    Seems like a waste to lose the benefits of a provisioning/management system for the sake of custom compiling libraries, OFED, MPI, etc. to gain that last bit of optimized performance.

  2. @EmmEff

    The issue is that OFED doesn’t support the Fedora 9 kernels (or kernels 2.6.25 or later) at this point. So our choice is (for this customer who wants Fedora), to roll back the kernel to something OFED will support, or to fix the OFED component (ofa_kernel) for the forward port. Rolling back the kernel will be easier and less intrusive to the customer, as I am assuming that sometime in the not so distant future, OFED will support the 2.6.25 and later kernels.

    This is less of a performance issue, and more of a core support of functionality issue.

    The problem I ran into is that building our own version of the kernel (down-rev) using the spec file they provide, and selecting a vanilla build of 2.6.24 doesn’t work. It doesn’t work, as RPM appears to be doing things in an un-expected way. More to the point, the spec file is doing virtual backflips within it to handle multiple cases of things that, arguably, should not be handled there, and certainly not in that manner. Again, this is my opinion, and I am aware that many others disagree.

    The issue here is building the customized kernel is just short of a full-on nightmare. The only way I see to make it un-nightmare-ish, is to pull all the business logic out of the spec file, and use the spec file for precisely what it is supposed to be used for, repeatable packaging with a fixed build flow (but pull the business logic about that build out of the RPMs control so it doesn’t try to be so … helpful …).

    Yeah this is a rant, yeah, others may think kernel builds are easy as pie using distro supplied rpm spec files and one or two line change. I don’t see it. (and yes, I have looked at the howto’s which add a patch or two … but don’t try anything like a kernel down-rev).

    It should be easier, but it isn’t. The kernel itself has an rpm target that works quite well. All we need for this, is a make rpm_headers or rpm_devel and this would be IMO the right method to use.

    FWIW: I am seriously contemplating building this kernel outside of the RPM system, as a) it will work, b) everything will be installed properly.

Comments are closed.