From nobody Fri Sep 12 00:17:40 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9ACB0C61DA4 for ; Tue, 14 Feb 2023 18:58:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232520AbjBNS6W (ORCPT ); Tue, 14 Feb 2023 13:58:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229608AbjBNS6U (ORCPT ); Tue, 14 Feb 2023 13:58:20 -0500 Received: from wp530.webpack.hosteurope.de (wp530.webpack.hosteurope.de [80.237.130.52]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B29F2FC; Tue, 14 Feb 2023 10:58:14 -0800 (PST) Received: from ip4d148da6.dynamic.kabel-deutschland.de ([77.20.141.166] helo=truhe.fritz.box); authenticated by wp530.webpack.hosteurope.de running ExIM with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) id 1pS0VG-0006UN-E5; Tue, 14 Feb 2023 19:58:10 +0100 From: Thorsten Leemhuis To: Jonathan Corbet Cc: Randy Dunlap , Lukas Bulwahn , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, regressions@lists.linux.dev, Greg KH , Jani Nikula Subject: [PATCH v2] docs: describe how to quickly build a trimmed kernel Date: Tue, 14 Feb 2023 19:58:09 +0100 Message-Id: <8cfcf069d48c1b8d7b83aafe0132f8dad0f1d0ea.1676400947.git.linux@leemhuis.info> X-Mailer: git-send-email 2.39.1 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-bounce-key: webpack.hosteurope.de;linux@leemhuis.info;1676401094;7a861e04; X-HE-SMSGID: 1pS0VG-0006UN-E5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a text explaining how to quickly build a kernel, as that's something users will often have to do when they want to report an issue or test proposed fixes. This is a huge and frightening task for quite a few users these days, as many rely on pre-compiled kernels and have never built their own. They find help on quite a few websites explaining the process in various ways, but those howtos often omit important details or make things too hard for the 'quickly build just for testing' case that 'localmodconfig' is really useful for. Hence give users something at hand to guide them, as that makes it easier for them to help with testing, debugging, and fixing the kernel. To keep the complexity at bay, the document explicitly focuses on how to compile the kernel on commodity distributions running on commodity hardware. People that deal with less common distributions or hardware will often know their way around already anyway. The text describes a few oddities of Arch and Debian that were found by the author and a few volunteers that tested the described procedure. There are likely more such quirks that need to be covered as well as a few things the author will have missed -- but one has to start somewhere. The document heavily uses anchors and links to them, which makes things slightly harder to read in the source form. But the intended target audience is way more likely to read rendered versions of this text on pages like docs.kernel.org anyway -- and there those anchors and links allow easy jumps to the reference section and back, which makes the document a lot easier to work with for the intended target audience. Aspects relevant for bisection were left out on purpose, as that is a related, but in the end different use case. The rough plan is to have a second document with a similar style to cover bisection. The idea is to reuse a few bits from this document and link quite often to entries in the reference section with the help of the anchors in this text. Signed-off-by: Thorsten Leemhuis --- Lo! Here is v2 of the text with some big changes after input from the community. Ciao, Thorsten v2: - default to a shallow clone - rename to "How to quickly build a trimmed Linux kernel" - use a localversion file instead of patching Makefile - various small improvements v1: https://lore.kernel.org/regressions/fabdb44fa44db2531f0dbe5e88545c49dfb8704= 0.1675252073.git.linux@leemhuis.info/ - inital version P.S.: a rendered version of this text is temporarily available at: https://www.leemhuis.info/files/misc/How%20to%20quickly%20build%20a%20trimm= ed%20Linux%20kernel%20%e2%80%94%20The%20Linux%20Kernel%20documentation.html --- Documentation/admin-guide/index.rst | 1 + .../quickly-build-trimmed-linux.rst | 1047 +++++++++++++++++ MAINTAINERS | 1 + 3 files changed, 1049 insertions(+) create mode 100644 Documentation/admin-guide/quickly-build-trimmed-linux.r= st diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guid= e/index.rst index f475554382e2..f796d65ba1f8 100644 --- a/Documentation/admin-guide/index.rst +++ b/Documentation/admin-guide/index.rst @@ -37,6 +37,7 @@ problems and bugs in particular. reporting-issues reporting-regressions security-bugs + quickly-build-trimmed-linux bug-hunting bug-bisect tainted-kernels diff --git a/Documentation/admin-guide/quickly-build-trimmed-linux.rst b/Do= cumentation/admin-guide/quickly-build-trimmed-linux.rst new file mode 100644 index 000000000000..6d7b482790af --- /dev/null +++ b/Documentation/admin-guide/quickly-build-trimmed-linux.rst @@ -0,0 +1,1047 @@ +.. SPDX-License-Identifier: (GPL-2.0+ OR CC-BY-4.0) +.. [see the bottom of this file for redistribution information] + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +How to quickly build a trimmed Linux kernel +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +This guide explains how to swiftly build Linux kernels that are ideal for +testing purposes, but perfectly fine for day-to-day use, too. + +The essence of the process (aka 'TL;DR') +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +*[If you are new to compiling Linux, ignore this TLDR and head over to the= next +section below: it contains a step-by-step guide, which is more detailed, b= ut +still brief and easy to follow; that guide and its accompanying reference +section also mention alternatives, pitfalls, and additional aspects, all of +which might be relevant for you.]* + +If your platform uses techniques like Secure Boot, prepare the system to p= ermit +starting self-compiled Linux kernels; install compilers and everything else +needed for building Linux; make sure to have 15 Gigabyte free space in you= r home +directory. Now run the following commands to download fresh Linux mainline +sources, which you then use to configure, build and install your own kerne= l:: + + mkdir ~/linux ~/linux/sources ~/linux/build + git clone --depth 1 -b master \ + https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git ~/l= inux/sources/ + cd ~/linux/sources/ + # Hint: if you want to apply patches, do it at this point; see below f= or details. + # Hint: it's recommended to tag your build at this point; see below fo= r details. + yes "" | make O=3D~/linux/build/ localmodconfig + # Hint: at this point you might want to adjust the build configuration; + # you'll have to, if you are running Debian. See below for details. + make -j $(nproc --all) O=3D~/linux/build/ + # Note: on many commodity distributions the next command suffices, but= on Arch + # Linux, its derivatives, and some others it does not. See below for= details. + command -v installkernel && sudo make O=3D~/linux/build/ modules_insta= ll install + reboot + +If you later want to build a newer mainline snapshot, use commands like th= ese:: + + cd ~/linux/sources/ + git fetch --depth 1 + # Note: the next command will discard any changes you did to the code: + git checkout --force --detach origin/master + # Reminder: if you want to (re)apply patches, do it at this point. + # Reminder: you might want to add or modify a build tag at this point. + make O=3D~/linux/build/ olddefconfig + make -j $(nproc --all) O=3D~/linux/build/ + # Reminder: the next command on some distributions does not suffice. + command -v installkernel && sudo make O=3D~/linux/build/ modules_insta= ll install + reboot + +Step-by-step guide +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Compiling your own Linux kernel is easy in principle. There are various wa= ys to +do it. Which of them actually work and is the best depends on the circumst= ances. + +This guide describes a way perfectly suited for those who want to quickly +install Linux from sources without being bothered by complicated details; = the +goal is to cover everything typically needed on mainstream Linux distribut= ions +running on commodity PC or server hardware. + +The described approach is great for testing purposes, for example to try a +proposed fix or check if the latest codebase still contains a problem. +Nonetheless, kernels built this way are also totally fine for day-to-day u= se +while at the same time being easy to keep up to date. + +The following steps describe the important aspects of the process; a +comprehensive reference section later explains each of them in more detail= . It +sometimes also describes alternative approaches, pitfalls, as well as erro= rs +that might occur at a particular point -- and how to then get things rolli= ng +again. + +.. _backup_sbs: + + * Create a fresh backup and put system repair and restore tools at hand, = just + to be prepared for the unlikely case that something might go sideways. + + [:ref:`details`] + +.. _secureboot_sbs: + + * On platforms with 'Secure Boot' or similar techniques, prepare everythi= ng to + ensure the system will permit your self-compiled kernel to boot later. = The + quickest and easiest way to achieve this on commodity x86 systems is to= turn + such techniques off in the BIOS setup utility; alternatively remove the= ir + restrictions through a process initiated by ``mokutil --disable-validat= ion``. + + [:ref:`details`] + +.. _buildrequires_sbs: + + * Install all software required to build a Linux kernel. Often you will n= eed: + 'bc', 'binutils' ('ld' et al.), 'bison', 'flex', 'gcc', 'git', 'openssl= ', + 'pahole', 'perl', and the development headers for 'libelf' and 'openssl= '. The + reference section shows how to quickly install those on various popular= Linux + distributions. + + [:ref:`details`] + +.. _diskspace_sbs: + + * Ensure to have enough free space for building and installing Linux. For= the + latter 150 Megabyte in /lib/ and 100 in /boot/ are a safe bet. For stor= ing + sources and build artifacts 15 Gigabyte in your home directory should + typically suffice. If you have less available, be sure to check the ref= erence + section for the steps about downloading the Linux sources and adjusting= your + kernels build configuration: they mention tricks that reduce the amount= of + required space in /home/ to around 3,5 Gigabyte. + + [:ref:`details`] + +.. _directories_sbs: + + * Create directories for sources and build artifacts:: + + mkdir ~/linux ~/linux/sources ~/linux/build + + [:ref:`details`] + +.. _sources_sbs: + + * Retrieve the sources of the Linux version you intend to build; then cha= nge + into the directory holding them, as all further commands in this guide = are + meant to be executed from there. + + *[Note: the following paragraphs describe how to retrieve the sources by + partially cloning the Linux stable git repository. This is called a sha= llow + clone. The reference section explains two alternatives:* :ref:`packaged + archives` *and* :ref:`a full git clone` = *; + prefer the latter, if downloading a lot of data does not bother you, as= that + will avoid some* :ref:`peculiar characteristics of shallow clones the + reference section explains` *.]* + + Execute the following command to retrieve a fresh mainline codebase:: + + git clone --no-checkput --depth 1 -b master \ + https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git \ + ~/linux/sources/ + cd ~/linux/sources/ + + If you want to access recent mainline releases and pre-releases, deepen= you + clone's history to the oldest version you are interested in:: + + git fetch --shallow-exclude=3Dv6.0 + + In case you want to access a stable/longterm release (say v6.1.5), simp= ly add + the branch holding that series; afterwards fetch the history up to the + mainline version that started the series (v6.1) or is older:: + + git remote set-branches --add origin linux-6.1.y + git fetch --shallow-exclude=3Dv6.0 + + Now checkout the code you are interested in. If you just performed the + initial clone, you will be able to check out a fresh mainline codebase,= which + is ideal for checking whether developers already addressed an issue:: + + git checkout --detach origin/master + + If you deepened your clone, you instead of ``origin/master`` can specif= y the + version you deepened to; any later releases like ``v6.1`` or pre-releas= e like + ``v6.2-rc1`` will work, too. Stable or longterm versions like ``v6.1.5`= ` work + just the same, if you added the appropriate stable/longterm branches as + described. + + [:ref:`details`] + +.. _patching_sbs: + + * In case you want to apply a kernel patch, do so now. Often a command li= ke + this will do the trick:: + + patch -p1 < ../proposed-fix.patch + + If the ``-p1`` is actually needed, depends on how the patch was created= ; in + case it does not apply thus try without it. + + If you cloned the sources with git and anything goes sideways, run ``git + reset --hard`` to undo any changes to the sources. + + [:ref:`details`] + +.. _tagging_sbs: + + * If you patched your kernel or have one of the same version installed al= ready, + better tag the one you are about to build uniquely:: + + echo "-proposed_fix" > ~/linux/build/localversion + + As a result running ``uname -r`` under your kernel later will print som= ething + like '6.1-rc4-proposed_fix'. + + [:ref:`details`] + + .. _configuration_sbs: + + * Create the build configuration for your kernel based on an existing + configuration. + + If you already prepared such a '.config' file yourself, copy it to + ~/linux/build/ and run ``make O=3D~/linux/build/ olddefconfig``. + + Use the same command, if your distribution or somebody else already tai= lored + your running kernel to your or your hardware's needs: the make target + 'olddefconfig' will then try to use that kernel's .config as base. + + Using this make target is fine for everybody else, too -- but you often= can + save a lot of time by using this command instead:: + + yes "" | make O=3D~/linux/build/ localmodconfig + + This will also try to pick your distribution's kernel as base, but then + disable modules for any features apparently superfluous for your setup.= This + will later reduce the compile time enormously, especially if you are ru= nning + an universal kernel from a mainstream Linux distribution. + + There is a catch: the make target 'localmodconfig' will disable kernel + features you have not actually utilized through some action or program = since + you booted the system. You can reduce or even eliminate that risk by us= ing + tricks outlined in the reference section; for quick testing purposes th= at + risk is often negligible, but it is an aspect you want to keep in mind = in + case your kernel behaves oddly. + + [:ref:`details`] + +.. _configmods_sbs: + + * Check if you might want to or have to adjust some kernel configuration + options: + + * Evaluate how you want to handle debug symbols. Enable them, if you lat= er + might need to decode a stack trace found for example in a 'panic', 'Oo= ps', + 'warning', or 'BUG'; on the other hand disable them, if you are short = on + storage space or prefer a smaller kernel binary. See the reference sec= tion + for details on how to do either. If neither applies, it will likely be= fine + to simply not bother with this. [:ref:`details`] + + * Are you running Debian? Then to avoid known problems you need to perfo= rm + additional adjustments explained in the reference section. + [:ref:`details`]. + +.. _build_sbs: + + * Build the image and the modules of your kernel:: + + make -j $(nproc --all) O=3D~/linux/build/ + + [:ref:`details`] + +.. _install_sbs: + + * Now install your kernel:: + + command -v installkernel && sudo make O=3D~/linux/build/ modules_inst= all install + + Often all left for you to do afterwards is a ``reboot``, as many commod= ity + Linux distributions will create an initramfs (also known as initrd) and= an + entry for your kernel in your boot-loader's configuration when you exec= ute + above command; but on some distributions you have to take care of these= two + steps manually for reasons the reference section explains. + + On a few distributions like Arch Linux and its derivatives the above co= mmand + does nothing at all; in that case you have to manually install your ker= nel, + as outlined in the reference section. + + [:ref:`details`] + +.. _another_sbs: + + * To later build another kernel you need similar steps, but sometimes sli= ghtly + different commands. + + First, switch back into the sources tree:: + + cd ~/linux/sources + + In case you want to build a version from a stable/longterm series you h= ave + not used yet (say 6.2.y), tell git to track it:: + + git remote set-branches --add origin linux-6.2.y + + Now fetch the latest upstream changes; you again need to specify the ea= rliest + version you care about, as git otherwise might retrieve the entire comm= it + history:: + + git fetch origin --shallow-exclude=3Dv6.1 + + If you modified the sources (for example by applying a patch), you now = need + to discard those modifications; that's because git otherwise will not b= e able + to switch to the sources of another version due to potential conflicting + changes:: + + git reset --hard + + Now checkout the version you are interested in:: + + git checkout --detach origin/master + + At this point you might want to patch the sources again or set/modify a= build + tag, as explained earlier; afterwards adjust the build configuration to= the + new codebase and build your next kernel:: + + # reminder: if you want to apply patches, do it at this point + # reminder: you might want to update your build tag at this point + make O=3D~/linux/build/ olddefconfig + make -j $(nproc --all) O=3D~/linux/build/ + + Install this kernel as outlined already:: + + command -v installkernel && sudo make O=3D~/linux/build/ modules_inst= all install + + [:ref:`details`] + +.. _uninstall_sbs: + + * Your kernel is easy to remove later, as its parts are only stored in two + places and clearly identifiable by the kernel's release name. Just ensu= re to + not delete the kernel you are running, as that might render your system + unbootable. + + Start by deleting the directory holding your kernel's modules, which is= named + after its release name -- '6.0.1-foobar' in the following example:: + + sudo rm -rf /lib/modules/6.0.1-foobar + + Now try the following command, which on some distributions will delete = all + other kernel files installed and remove the kernel's entry from the boot + loader configuration:: + + command -v kernel-install && sudo kernel-install -v remove 6.0.1-foob= ar + + If that command does not output anything or fails, see the reference se= ction; + do the same if any files named '*6.0.1-foobar*' remain in /boot/. + + [:ref:`details`] + +.. _submit_improvements: + +Did you run into trouble following any of the above steps that is not clea= red up +by the reference section below? Or do you have ideas how to improve the te= xt? +Then please take a moment of your time and let the maintainer of this docu= ment +know by email (Thorsten Leemhuis ), ideally while CCi= ng the +Linux docs mailing list (linux-doc@vger.kernel.org). Such feedback is vita= l to +improve this document further, which is in everybody's interest, as it will +enable more people to master the task described here. + +Reference section for the step-by-step guide +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +This section holds additional information for each of the steps in the abo= ve +guide. + +.. _backup: + +Prepare for emergencies +----------------------- + + *Create a fresh backup and put system repair and restore tools at hand* + [:ref:`... `] + +Remember, you are dealing with computers, which sometimes do unexpected th= ings +-- especially if you fiddle with crucial parts like the kernel of an opera= ting +system. That's what you are about to do in this process. Hence, better pre= pare +for something going sideways, even if that should not happen. + +[:ref:`back to step-by-step guide `] + +.. _secureboot: + +Dealing with techniques like Secure Boot +---------------------------------------- + + *On platforms with 'Secure Boot' or similar techniques, prepare everyth= ing to + ensure the system will permit your self-compiled kernel to boot later.* + [:ref:`... `] + +Many modern systems allow only certain operating systems to start; they th= us by +default will reject booting self-compiled kernels. + +You ideally deal with this by making your platform trust your self-built k= ernels +with the help of a certificate and signing. How to do that is not described +here, as it requires various steps that would take the text too far away f= rom +its purpose; but you will find many pages on the web that explain this in +detail. + +Temporarily disabling solutions like Secure Boot is another way to make yo= ur own +Linux boot. On commodity x86 systems it is possible to do this in the BIOS= Setup +utility; the steps to do so are not described here, as they greatly vary b= etween +machines. + +On mainstream x86 Linux distributions there is a third and universal optio= n: +disable all Secure Boot restrictions for your Linux environment. You can +initiate this process by running ``mokutil --disable-validation``; this wi= ll +tell you to create a one-time password, which is safe to write down. Now +restart; right after your BIOS performed all self-tests the bootloader Shi= m will +show a blue box with a message 'Press any key to perform MOK management'. = Hit +some key before the countdown exposes. This will open a menu and choose 'C= hange +Secure Boot state' there. Shim's 'MokManager' will now ask you to enter th= ree +randomly chosen characters from the one-time password specified earlier. O= nce +you provided them, confirm that you really want to disable the validation. +Afterwards, permit MokManager to reboot the machine. + +[:ref:`back to step-by-step guide `] + +.. _buildrequires: + +Install build requirements +-------------------------- + + *Install all software required to build a Linux kernel.* + [:ref:`...`] + +The kernel is pretty stand-alone, but besides tools like the compiler you = will +sometimes need a few libraries to build one. How to install everything nee= ded +depends on your Linux distribution and the configuration of the kernel you= are +about to build. + +Here are a few examples what you typically need on some mainstream +distributions: + + * Debian, Ubuntu, and derivatives:: + + sudo apt install bc binutils bison dwarves flex gcc git make openssl \ + pahole perl-base libssl-dev libelf-dev + + * Fedora and derivatives:: + + sudo dnf install binutils /usr/include/{libelf.h,openssl/pkcs7.h} \ + /usr/bin/{bc,bison,flex,gcc,git,openssl,make,perl,pahole} + + * openSUSE and derivatives:: + + sudo zypper install bc binutils bison dwarves flex gcc git make perl-= base \ + openssl openssl-devel libelf-dev + +In case you wonder why these lists include openssl and its development hea= ders: +they are needed for the Secure Boot support, which many distributions enab= le in +their kernel configuration for x86 machines. + +Sometimes you will need tools for compression formats like bzip2, gzip, lz= 4, +lzma, lzo, xz, or zstd as well. + +You might need additional libraries and their development headers in case = you +perform tasks not covered in this guide. For example, zlib will be needed = when +building kernel tools from the tools/ directory; make targets like 'menuco= nfig' +or 'xconfig' will require development headers for ncurses or Qt. + +[:ref:`back to step-by-step guide `] + +.. _diskspace: + +Space requirements +------------------ + + *Ensure to have enough free space for building and installing Linux.* + [:ref:`... `] + +The numbers mentioned are rough estimates with a big extra charge to be on= the +safe side, so often you will need less. + +If you have space constraints, remember to read the reference section for = the +steps :ref:`Downloading the Linux sources` and :ref:`Debug symbol= s in +the section about configuration adjustments' `: both mention t= ricks +that reduce the consumed disk space by quite a lot. + +[:ref:`back to step-by-step guide `] + + +.. _directories: + +Create dedicated directories +---------------------------- + + *Create directories for sources and build artifacts:* + [:ref:`...`] + +You could build your kernel directly in the source tree, but it is good pr= actice +to keep it clean by storing the build artifacts separately. That's why this +guide recommends creating a directory for the latter, which you hence must +specify in later commands. Most of the time this is done by passing +``O=3D~/linux/build/`` to make. + +[:ref:`back to step-by-step guide `] + +.. _sources: + +Download the sources +-------------------- + + *Retrieve the sources of the Linux version you intend to build.* + [:ref:`...`] + +The step-by-step guide outlines how to retrieve Linux' sources using a sha= llow +git clone. There is :ref:`more to tell about this method`= and +two alternate ways worth describing: :ref:`packaged archives` +and :ref:`a full git clone`. And the aspects ':ref:`wouldn't= it +be wiser to use a proper pre-release than the latest mainline code +`' and ':ref:`how to get an even fresher mainline codeba= se +`' need elaboration, too. + +[:ref:`back to step-by-step guide `] + +.. _sources_shallow: + +Noteworthy characteristics of shallow clones +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The step-by-step guide uses a shallow clone, as it is the best solution fo= r most +of this document's target audience. There are a few aspects of this approa= ch +worth mentioning: + + * This document in most places uses ``git fetch`` with ``--shallow-exclud= e=3D`` + to specify the tag of the earliest version you care about. You alternat= ively + cat use the parameter ``--shallow-since=3D`` to specify an absolute (say + ``'2023-07-15'``) or relative (``'12 months'``) date to define the dept= h of + the history you want to download. As a second alternative, you can also + specify a certain depth explicitly with a parameter like ``--depth=3D1`= `, + unless you add branches for stable/longterm kernels. + + * When running ``git fetch``, remember to always specify the oldest versi= on, + the time you care about, or an explicit depth as shown in the step-by-s= tep + guide. Otherwise you will risk downloading nearly the entire git histor= y, + which will consume quite a bit of time and bandwidth while also stressi= ng the + servers. + + Note, you do not have to use the same version or date all the time. But= when + you change it over time, git will deepen or flatten the history to the + specified point. That allows you to retrieve versions you initially tho= ught + you did not need -- or it will discard the sources of older versions, f= or + example in case you want to free up some disk space. The latter will ha= ppen + automatically when using ``--shallow-since=3D`` or + ``--depth=3D``. + + * Be warned, when deepening your clone you might encounter an error like + 'fatal: error in object: unshallow cafecaca0c0dacafecaca0c0dacafecaca0c= 0da'. + In that case run ``git repack -d`` and try again`` + + * In case you want to revert changes from a certain version (say Linux 6.= 3) or + perform a bisection (v6.2..v6.3), better tell ``git fetch`` to retrieve + objects up to three versions earlier (e.g. 6.0): ``git describe`` will = then + be able to describe most commits just like it would in a full git clone. + +[:ref:`back to step-by-step guide `] [:ref:`back to section i= ntro `] + +.. _sources_archive: + +Downloading the sources using a packages archive +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +People new to compiling Linux often assume downloading an archive via the +front-page of https://kernel.org is the best approach to retrieve Linux' +sources. It actually can be, if you are certain to build just one particul= ar +kernel version without changing any code. Thing is: you might be sure this= will +be the case, but in practice it often will turn out to be a wrong assumpti= on. + +That's because when reporting or debugging an issue developers will often = ask to +give another version a try. They also might suggest temporarily undoing a = commit +with ``git revert`` or might provide various patches to try. Sometimes rep= orters +will also be asked to use ``git bisect`` to find the change causing a prob= lem. +These things rely on git or are a lot easier and quicker to handle with it. + +A shallow clone also does not add any significant overhead. For example, w= hen +you use ``git clone --depth=3D1`` to create a shallow clone of the latest = mainline +codebase git will only retrieve a little more data than downloading a comp= ressed +packaged archive would. + +A shallow clone therefore is often the better choice. If you nevertheless = want +to use a packaged source archive, download one via kernel.org; afterwards +extract its content to ``~/linux/`` and change to the directory created du= ring +extraction. The rest of the step-by-step guide will work just fine, apart = from +things that rely on git -- but this mainly concerns the section on success= ive +builds of other versions. + +[:ref:`back to step-by-step guide `] [:ref:`back to section i= ntro `] + +.. _sources_full: + +Downloading the sources using a full git clone +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If downloading and storing a lot of data (~4,4 Gigabyte as of early 2023) = is +nothing that bothers you, instead of a shallow clone perform a full git cl= one +instead. You then will avoid the specialties mentioned above and will have= all +versions and individual commits at hand at any time:: + + curl -L https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.g= it/clone.bundle -o ~/linux/linux-stable.git.bundle + git clone clone.bundle ~/linux/sources/ + rm ~/linux/linux-stable.git.bundle + cd ~/linux/sources/ + git remote set-url origin + https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git + git fetch --all + git checkout --detach origin/master + +[:ref:`back to step-by-step guide `] [:ref:`back to section i= ntro `] + +.. _sources_snapshot: + +Proper pre-releases (RCs) vs. latest mainline +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When cloning the sources using git and checking out origin/master, you oft= en +will retrieve a codebase that is somewhere between the latest and the next +release or pre-release. This almost always is the code you want when giving +mainline a shot: pre-releases like v6.1-rc5 are in no way special, as they= do +not get any significant extra testing before being published. + +There is one exception: you might want to stick to the latest mainline rel= ease +(say v6.1) before its successor's first pre-release (v6.2-rc1) is out. Tha= t is +because compiler errors and other problems are more likely to occur during= this +time, as mainline then is in its 'merge window': a usually two week long p= hase, +in which the bulk of the changes for the next release is merged. + +[:ref:`back to step-by-step guide `] [:ref:`back to section i= ntro `] + +.. _sources_fresher: + +Avoiding the mainline lag +~~~~~~~~~~~~~~~~~~~~~~~~~ + +The explanations for both the shallow clone and the full clone both retrie= ve the +code from the Linux stable git repository. That makes things simpler for t= his +document's audience, as it allows easy access to both mainline and +stable/longterm releases. This approach has just one downside: + +Changes merged into the mainline repository are only synced to the master = branch +of the Linux stable repository every few hours. This lag most of the time= is +not something to worry about; but in case you really need the latest code,= just +add the mainline repo as additional remote and checkout the code from ther= e:: + + git remote add mainline https://git.kernel.org/pub/scm/linux/kernel/gi= t/torvalds/linux.git + git fetch + git checkout --detach mainline/master + +When doing this with a shallow clone, remember to call ``git fetch`` with = one +of the parameters described earlier to limit the depth. + +[:ref:`back to step-by-step guide `] [:ref:`back to section i= ntro `] + +.. _patching: + +Patch the sources (optional) +---------------------------- + + *In case you want to apply a kernel patch, do so now.* + [:ref:`...`] + +This is the point where you might want to patch your kernel -- for example= when +a developer proposed a fix and asked you to check if it helps. The step-by= -step +guide already explains everything crucial here. + +[:ref:`back to step-by-step guide `] + +.. _tagging: + +Tagging this kernel build (optional, often wise) +------------------------------------------------ + + *If you patched your kernel or already have that kernel version installe= d, + better tag your kernel by extending its release name:* + [:ref:`...`] + +Tagging your kernel will help avoid confusion later, especially when you p= atched +your kernel. Adding an individual tag will also ensure the kernel's image = and +its modules are installed in parallel to any existing kernels. + +There are various ways to add such a tag. The step-by-step guide realizes = one by +creating a 'localversion' file in your build directory from which the kern= el +build scripts will automatically pick up the tag. You can later change tha= t file +to use a different tag in subsequent builds or simply remove that file to = dump +the tag. + +[:ref:`back to step-by-step guide `] + +.. _configuration: + +Define the build configuration for your kernel +---------------------------------------------- + + *Create the build configuration for your kernel based on an existing + configuration.* [:ref:`... `] + +There are various aspects for this steps that require a more careful +explanation: + +Pitfalls when using another configuration file as base +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Make targets like localmodconfig and olddefconfig share a few common snare= s you +want to be aware of: + + * These targets will reuse a kernel build configuration in your build dir= ectory + (e.g. '~/linux/build/.config'), if one exists. In case you want to star= t from + scratch you thus need to delete it. + + * The make targets try to find the configuration for your running kernel + automatically, but might choose poorly. A line like '# using defaults f= ound + in /boot/config-6.0.7-250.fc36.x86_64' or 'using config: + '/boot/config-6.0.7-250.fc36.x86_64' tells you which file they picked. = If + that is not the intended one, simply store it as '~/linux/build/.config' + before using these make targets. + + * Unexpected things might happen if you try to use a config file prepared= for + one kernel (say v6.0) on an older generation (say v5.15). In that case = you + might want to use a configuration as base which your distribution utili= zed + when they used that or an slightly older kernel version. + +Influencing the configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The make target olddefconfig and the ``yes "" |`` used when utilizing +localmodconfig will set any undefined build options to their default value= . This +among others will disable many kernel features that were introduced after = your +base kernel was released. + +If you want to set these configurations options manually, use ``oldconfig`` +instead of ``olddefconfig`` or omit the ``yes "" |`` when utilizing +localmodconfig. Then for each undefined configuration option you will be a= sked +how to proceed. In case you are unsure what to answer, simply hit 'enter' = to +apply the default value. + +Big pitfall when using localmodconfig +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As explained briefly in the step-by-step guide already: with localmodconfi= g it +can easily happen that your self-built kernel will lack modules for tasks = you +did not perform before utilizing this make target. That's because those ta= sks +require kernel modules that are normally autoloaded when you perform that = task +for the first time; if you didn't perform that task at least once before u= sing +localmodonfig, the latter will thus assume these modules are superfluous a= nd +disable them. + +You can try to avoid this by performing typical tasks that often will auto= load +additional kernel modules: start a VM, establish VPN connections, loop-mou= nt a +CD/DVD ISO, mount network shares (CIFS, NFS, ...), and connect all external +devices (2FA keys, headsets, webcams, ...) as well as storage devices with= file +systems you otherwise do not utilize (btrfs, ext4, FAT, NTFS, XFS, ...). B= ut it +is hard to think of everything that might be needed -- even kernel develop= ers +often forget one thing or another at this point. + +Do not let that risk bother you, especially when compiling a kernel only f= or +testing purposes: everything typically crucial will be there. And if you f= orget +something important you can turn on a missing feature later and quickly ru= n the +commands to compile and install a better kernel. + +But if you plan to build and use self-built kernels regularly, you might w= ant to +reduce the risk by recording which modules your system loads over the cour= se of +a few weeks. You can automate this with `modprobed-db +`_. Afterwards use ``LSMOD=3D`` to +point localmodconfig to the list of modules modprobed-db noticed being use= d:: + + yes "" | make O=3D~/linux/build/ LSMOD=3D"${HOME}"/.config/modprobed.d= b localmodconfig + +Remote building with localmodconfig +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you want to use localmodconfig to build a kernel for another machine, r= un +``lsmod > lsmod_foo-machine`` on it and transfer that file to your build h= ost. +Now point the build scripts to the file like this: ``yes "" | make +O=3D~/linux/build/ LSMOD=3D~/lsmod_foo-machine localmodconfig``. Note, in = this case +you likely want to copy a base kernel configuration from the other machine= over +as well and place it as .config in your build directory. + +[:ref:`back to step-by-step guide `] + +.. _configmods: + +Adjust build configuration +-------------------------- + + *Check if you might want to or have to adjust some kernel configuration + options:* + +Depending on your needs you at this point might want or have to adjust some +kernel configuration options. + +.. _configmods_debugsymbols: + +Debug symbols +~~~~~~~~~~~~~ + + *Evaluate how you want to handle debug symbols.* + [:ref:`...`] + +Most users do not need to care about this, it's often fine to leave everyt= hing +as it is; but you should take a closer look at this, if you might need to = decode +a stack trace or want to reduce space consumption. + +Having debug symbols available can be important when your kernel throws a +'panic', 'Oops', 'warning', or 'BUG' later when running, as then you will = be +able to find the exact place where the problem occurred in the code. But +collecting and embedding the needed debug information takes time and consu= mes +quite a bit of space: in late 2022 the build artifacts for a typical x86 k= ernel +configured with localmodconfig consumed around 5 Gigabyte of space with de= bug +symbols, but less than 1 when they were disabled. The resulting kernel ima= ge and +the modules are bigger as well, which increases load times. + +Hence, if you want a small kernel and are unlikely to decode a stack trace +later, you might want to disable debug symbols to avoid above downsides:: + + ./scripts/config --file ~/linux/build/.config -d DEBUG_INFO \ + -d DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT -d DEBUG_INFO_DWARF4 \ + -d DEBUG_INFO_DWARF5 -e CONFIG_DEBUG_INFO_NONE + make O=3D~/linux/build/ olddefconfig + +You on the other hand definitely want to enable them, if there is a decent +chance that you need to decode a stack trace later (as explained by 'Decode +failure messages' in Documentation/admin-guide/tainted-kernels.rst in more +detail):: + + ./scripts/config --file ~/linux/build/.config -d DEBUG_INFO_NONE -e DE= BUG_KERNEL + -e DEBUG_INFO -e DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT -e KALLSYMS -e K= ALLSYMS_ALL + make O=3D~/linux/build/ olddefconfig + +Note, many mainstream distributions enable debug symbols in their kernel +configurations -- make targets like localmodconfig and olddefconfig thus w= ill +often pick that setting up. + +[:ref:`back to step-by-step guide `] + +.. _configmods_distros: + +Distro specific adjustments +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + *Are you running* [:ref:`... `] + +The following sections help you to avoid build problems that are known to = occur +when following this guide on a few commodity distributions. + +**Debian:** + + * Remove a stale reference to a certificate file that would cause your bu= ild to + fail:: + + ./scripts/config --file ~/linux/build/.config --set-str SYSTEM_TRUSTED= _KEYS '' + + Alternatively, download the needed certificate and make that configurat= ion + option point to it, as `the Debian handbook explains in more detail + `_ . + +[:ref:`back to step-by-step guide `] + +.. _build: + +Build your kernel +----------------- + + *Build the image and the modules of your kernel* [:ref:`... `] + +A lot can go wrong at this stage that will abort the build process. Someti= mes +this is caused by problems on your side, which you will often be able to f= ix +quickly; sometimes though the problem lies in the code and can only be fix= ed by +a developer. + +A close examination of the failure messages coupled with some research on = the +internet will often tell you which of the two it is. Before doing this, re= start +the build process like this:: + + make O=3D~/linux/build/ V=3D1 + +The ``V=3D1`` activates verbose output, which might be needed to see the a= ctual +error. To make it easier to spot, this command also omits the ``-j $(nproc +--all)`` used earlier to utilize every CPU core in the system for the job.= But +this parallelism results in some clutter when failures occur, which makes = the +actual error message harder to spot. + +Once the failure happens again, try to find the most crucial line describi= ng the +problem. Then search the internet for the most important and non-generic s= ection +of that line (say 4 to 8 words); avoid or remove anything that looks remot= ely +system-specific, like your username or local path names like ``~/linux/bui= ld/``. +First try your regular internet search engine with that string, afterwards +search Linux kernel mailing lists via `lore.kernel.org/all/ +`_. + +This most of the time will find something that will explain what is wrong;= quite +often one of the hits will provide a solution for your problem, too. If you +do not find anything that matches your problem, try again from a different= angle +by modifying your search terms or using another line from the printed error +messages. + +In the end, most trouble you are to run into has likely been encountered a= nd +reported by others already. That includes issues where the cause is not yo= ur +system, but lies the code. If you run into one of those, you might thus fi= nd a +solution (e.g. a patch) or workaround for your problem, too. + +[:ref:`back to step-by-step guide `] + +.. _install: + +Install your kernel +------------------- + + *Now install your kernel* [:ref:`... `] + +What you need to do after executing the command in the step-by-step guide +depends on the existence and the implementation of an ``installkernel`` +executable. Many commodity Linux distributions ship such a kernel installe= r in +``/sbin/`` that does everything needed, hence there is nothing left for you +except rebooting. But some distributions contain an installkernel that does +only part of the job -- and a few lack it completely and leave all the wor= k to +you. + +If ``installkernel`` is found, the kernel's build system will delegate the +actual installation of your kernel's image and related files to this execu= table. +On almost all Linux distributions it will store the image as '/boot/vmlinu= z- +' and put a 'System.map-' alongside it. Your kernel will thus be installed in parallel to any +existing ones, unless you already have one with exactly the same release n= ame. + +Installkernel on many distributions will afterwards generate an 'initramfs' +(often also called 'initrd'), which commodity distributions rely on for bo= oting; +hence be sure to keep the order of the two make targets used in the step-b= y-step +guide, as things will go sideways if you install your kernel's image befor= e its +modules. Often installkernel will then add your kernel to the boot loader +configuration, too. You have to take care of one or both of these tasks +yourself, if your distributions installkernel doesn't handle them. + +A few distributions like Arch Linux and its derivatives totally lack an +installkernel executable. On those just install the modules using the kern= el's +build system and then install the image and the System.map file manually:: + + sudo make O=3D~/linux/build/ modules_install + sudo install -m 0600 ~/linux/build/$(make O=3D~/linux/build/ -s image= _name) \ + /boot/vmlinuz-$(make O=3D~/linux/build/ -s kernelrelease) + sudo install -m 0600 ~/linux/build/System.map \ + /boot/System.map-$(make O=3D~/linux/build/ -s kernelrelease) + +If your distribution boots with the help of an initramfs, now generate one= for +your kernel using the tools your distribution provides for this process. +Afterwards add your kernel to your boot-loader configuration and reboot. + +[:ref:`back to step-by-step guide `] + +.. _another: + +Another round later +------------------- + + *To later build another kernel you need similar, but sometimes slightly + different commands* [:ref:`... `] + +The process to build later kernels is similar, but at some points slightly +different. You for example do not want to use 'localmodconfig' for succeed= ing +kernel builds, as you already created a trimmed down configuration you wan= t to +use from now on. Hence instead just use ``oldconfig`` or ``olddefconfig`` = to +adjust your build configurations to the needs of the kernel version you are +about to build. + +If you created a shallow-clone with git, remember what the :ref:`section t= hat +explained the setup described in more detail `: you need to use a +slightly different ``git fetch`` command and when switching to another ser= ies +need to add an additional remote branch. + +[:ref:`back to step-by-step guide `] + +.. _uninstall: + +Uninstall the kernel later +-------------------------- + + *All parts of your installed kernel are identifiable by its release name= and + thus easy to remove later.* [:ref:`... `] + +Do not worry installing your kernel manually and thus bypassing your +distribution's packaging system will totally mess up your machine: all par= ts of +your kernel are easy to remove later, as files are stored in two places on= ly and +normally identifiable by the kernel's release name. + +One of the two places is a directory in /lib/modules/, which holds the mod= ules +for each installed kernel. This directory is named after the kernel's rele= ase +name; hence, to remove all modules for one of your kernels, simply remove = its +modules directory in /lib/modules/. + +The other place is /boot/, where typically one to five files will be placed +during installation of a kernel. All of them usually contain the release n= ame in +their file name, but how many files and their name depends somewhat on your +distribution's installkernel executable (:ref:`see above `) and i= ts +initramfs generator. On some distributions the ``kernel-install`` command +mentioned in the step-by-step guide will remove all of these files for you= -- +and the entry for your kernel in the boot-loader configuration at the same= time, +too. On others you have to take care of these steps yourself. The following +command should interactively remove the two main files of a kernel with the +release name '6.0.1-foobar':: + + rm -i /boot/{System.map,vmlinuz}-6.0.1-foobar + +Now remove the belonging initramfs, which often will be called something l= ike +``/boot/initramfs-6.0.1-foobar.img`` or ``/boot/initrd.img-6.0.1-foobar``. +Afterwards check for other files in /boot/ that have '6.0.1-foobar' in the= ir +name and delete them as well. Now remove the kernel from your boot-loader's +configuration. + +Note, be very careful with wildcards like '*' when deleting files or direc= tories +for kernels manually: you might accidentally remove files of a 6.0.11 kern= el +when all you want is to remove 6.0 or 6.0.1. + +[:ref:`back to step-by-step guide `] + +.. _faq: + +FAQ +=3D=3D=3D + +Why does this 'how-to' not work on my system? +--------------------------------------------- + +As initially stated, this guide is 'designed to cover everything typically +needed [to build a kernel] on mainstream Linux distributions running on +commodity PC or server hardware'. The outlined approach despite this shoul= d work +on many other setups as well. But trying to cover every possible use-case = in one +guide would defeat its purpose, as without such a focus you would need doz= ens or +hundreds of constructs along the lines of 'in case you are having , you at this point have to do +'. Each of which would make the text longer, more +complicated, and harder to follow. + +That being said: this of course is a balancing act. Hence, if you think an +additional use-case is worth describing, suggest it to the maintainers of = this +document, as :ref:`described above `. + + +.. + end-of-content +.. + This document is maintained by Thorsten Leemhuis .= If + you spot a typo or small mistake, feel free to let him know directly and + he'll fix it. You are free to do the same in a mostly informal way if y= ou + want to contribute changes to the text -- but for copyright reasons ple= ase CC + linux-doc@vger.kernel.org and 'sign-off' your contribution as + Documentation/process/submitting-patches.rst explains in the section 'S= ign + your work - the Developer's Certificate of Origin'. +.. + This text is available under GPL-2.0+ or CC-BY-4.0, as stated at the top + of the file. If you want to distribute this text under CC-BY-4.0 only, + please use 'The Linux kernel development community' for author attribut= ion + and link this as source: + https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plai= n/Documentation/admin-guide/quickly-build-trimmed-linux.rst +.. + Note: Only the content of this RST file as found in the Linux kernel so= urces + is available under CC-BY-4.0, as versions of this text that were proces= sed + (for example by the kernel's build system) might contain content taken = from + files which use a more restrictive license. + diff --git a/MAINTAINERS b/MAINTAINERS index 7f86d02cb427..bfbed79c0ea1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -6329,6 +6329,7 @@ DOCUMENTATION REPORTING ISSUES M: Thorsten Leemhuis L: linux-doc@vger.kernel.org S: Maintained +F: Documentation/admin-guide/quickly-build-trimmed-linux.rst F: Documentation/admin-guide/reporting-issues.rst =20 DOCUMENTATION SCRIPTS base-commit: e076f253283c3e55a128fa9665c0e6cd8146948d --=20 2.39.1