Compiling the Kernel (Kernel 5.15)

This topic contains instructions for compiling the Linux kernel source in your Linux SDK product.
  1. Set up the environment macros: The LOCAL_VERSION environment variable appends -rt-tegra to the kernel version and modules, please ensure $NV_WORKSPACE is set to <top> of the SDK install directory.

    export ARCH=arm64
    export CROSS_COMPILE=${PWD}/toolchains/aarch64--glibc--stable-2022.03-1/bin/aarch64-buildroot-linux-gnu-
    export LOCALVERSION="-rt-tegra"
  2. Set the kernel source directory as the current working directory:

    cd $NV_WORKSPACE/drive-linux/kernel/source/oss_src
  3. Enter the following command to apply the rt-patches.

    cd kernel/scripts
    bash generic-rt-patch.sh apply-patches
    cd ../../ 
  4. Install the packages to ready the kernel building workspace:
    sudo apt-get update
    sudo apt-get -f -y install imagemagick graphviz dvipng python3-venv fonts-noto-cjk latexmk librsvg2-bin texlive-xetex flex bison
  5. Create an output directory and clean up.

    mkdir out-linux
    make -C kernel O=${PWD}/out-linux clean
    
  6. Configure the kernel to the standard kernel with the command. To build a production kernel instead of a standard kernel, please use tegra_prod_defconfig instead of defconfig.

    make -C kernel O=${PWD}/out-linux defconfig
  7. Build the kernel:
    make -j3 -C kernel O=${PWD}/out-linux
    Note: If the preceding command fails, enter the make command without the j<number> option.
  8. Build OOT modules:
    ln -s -f ${PWD}/nvidia-oot /tmp/nv-oot
    ln -s -f ${PWD}/nvgpu /tmp/nvgpu
    ln -s -f ${PWD}/hwpm /tmp/hwpm
    
    make -j $(nproc) -C ${PWD}/out-linux M=/tmp/hwpm/drivers/tegra/hwpm srctree.hwpm=/tmp/hwpm V=1 modules CONFIG_TEGRA_OOT_MODULE=m CONFIG_TEGRA_VIRTUALIZATION=y
    
    NV_BUILD_MODULE_DRV_MLNX=1 make -j $(nproc) -C ${PWD}/out-linux M=/tmp/nv-oot srctree.nvidia-oot=/tmp/nv-oot srctree.nvidia=/tmp/nv-oot srctree.hwpm=/tmp/hwpm  KBUILD_EXTRA_SYMBOLS=/tmp/hwpm/drivers/tegra/hwpm/Module.symvers V=1 modules CONFIG_TEGRA_OOT_MODULE=m CONFIG_TEGRA_VIRTUALIZATION=y
    
    NV_BUILD_MODULE_DRV_MLNX=1  make -j $(nproc) -C ${PWD}/out-linux M=/tmp/nvgpu/drivers/gpu/nvgpu srctree.nvidia-oot=/tmp/nv-oot srctree.nvidia=/tmp/nv-oot KBUILD_EXTRA_SYMBOLS=/tmp/nv-oot/Module.symvers V=1 modules CONFIG_TEGRA_OOT_MODULE=m CONFIG_TEGRA_VIRTUALIZATION=y
    Note: If the preceding command fails, enter the make command without the j<number> option.
  9. Build the MLNX Drivers using the steps below.
    Note: MLNX Drivers build only applies to standard kernel defconfig (i.e., not the production or recovery kernel defconfigs).
    cd $NV_WORKSPACE/drive-linux/kernel/source/oss_src/
    mkdir -p out-linux/src-rt/
    cp -al kernel ./out-linux/src-rt/
    ./kernel/scripts/build-module-mlnx.sh ./mlnx-drivers ./out-linux ./out-linux/src-rt/kernel ./out-linux-mlnx-build
  10. Consolidate the built kernel modules in the build directory with the following commands:
    export INSTALL_MOD_PATH=${PWD}/out-linux
    NV_BUILD_MODULE_DRV_MLNX=1 make -C kernel O=${PWD}/out-linux INSTALL_MOD_STRIP=1 modules_install
    NV_BUILD_MODULE_DRV_MLNX=1 make -C ${PWD}/out-linux M=/tmp/nv-oot INSTALL_MOD_STRIP=1 modules_install -j $(nproc)
    make -C ${PWD}/out-linux M=/tmp/hwpm/drivers/tegra/hwpm INSTALL_MOD_STRIP=1 modules_install 
    make -C ${PWD}/out-linux M=/tmp/nvgpu/drivers/gpu/nvgpu INSTALL_MOD_STRIP=1 modules_install
    rm /tmp/nv-oot
    rm /tmp/hwpm
    rm /tmp/nvgpu
  11. Recompile the display kernel modules.
    1. Unify headers and Module.symvers for display module compilation:
      cd $NV_WORKSPACE/drive-linux/kernel/source/oss_src
      cat nvidia-oot/Module.symvers >> out-linux/Module.symvers
      rsync -avzpq nvidia-oot/include/ out-linux/include
      rsync -avzpq nvidia-oot/drivers/gpu/host1x/include/ out-linux/drivers/gpu/host1x/include
    2. Untar the display source code.

      The NVIDIA-kernel-module-source-<Version>.tar.xz source code is supplied as a .xz file. Untar the file. Its location is <NV_WORKSPACE>/drive-linux_src/

      tar -xvf $NV_WORKSPACE/drive-linux_src/NVIDIA-kernel-module-source-TempVersion.tar.xz
    3. Compile the display kernel module.
      export IGNORE_PREEMPT_RT_PRESENCE=1
      cd NVIDIA-kernel-module-source-TempVersion
      make \
          modules \
          SYSSRC=$NV_WORKSPACE/drive-linux/kernel/source/oss_src/kernel \
          SYSOUT=$NV_WORKSPACE/drive-linux/kernel/source/oss_src/out-linux \
          SYSSRCHOST1X=$NV_WORKSPACE/drive-linux/kernel/source/oss_src/nvidia-oot/drivers/gpu/host1x/include\
          CC=${CROSS_COMPILE}gcc \
          LD=${CROSS_COMPILE}ld.bfd \
          AR=${CROSS_COMPILE}ar \
          CXX=${CROSS_COMPILE}g++ \
          OBJCOPY=${CROSS_COMPILE}objcopy \
          TARGET_ARCH=aarch64 \
          ARCH=arm64
    4. Copy the display modules to the module_install path.
      mkdir -p $NV_WORKSPACE/drive-linux/kernel/source/oss_src/out-linux/lib/modules/<kernel_version>/extra/opensrc-disp/
      cd kernel-open
      cp nvidia.ko nvidia-modeset.ko nvidia-drm.ko $NV_WORKSPACE/drive-linux/kernel/source/oss_src/out-linux/lib/modules/<kernel_version>/extra/opensrc-disp/
    5. Set the kernel source directory as the current working directory.
      cd $NV_WORKSPACE/drive-linux/kernel/source/oss_src
  12. To flash the built kernel, update the kernel Image, kernel modules, and then the filesystem:
    1. Copy the uncompressed (Image) kernel images to the top of the kernel directory with the following command:
      export PROD_SUFFIX=<str> # If using production kernel, set <str> to "_prod" else set to empty string ""
      sudo rm -fv $NV_WORKSPACE/drive-linux/kernel/preempt_rt${PROD_SUFFIX}/images/*
      mkdir -p  $NV_WORKSPACE/drive-linux/kernel/preempt_rt${PROD_SUFFIX}/images/
      sudo cp -v ${PWD}/out-linux/arch/arm64/boot/Image ${PWD}/out-linux/vmlinux ${PWD}/out-linux/System.map $NV_WORKSPACE/drive-linux/kernel/preempt_rt${PROD_SUFFIX}/images/
      CAUTION: Before copying the new kernel images, back up the default kernels provided.
    2. Copy the built modules to the SDK kernel modules path:
      sudo rm -rf $NV_WORKSPACE/drive-linux/kernel/preempt_rt${PROD_SUFFIX}/modules/*
      mkdir -p  $NV_WORKSPACE/drive-linux/kernel/preempt_rt${PROD_SUFFIX}/modules/
      sudo cp -a ${PWD}/out-linux/lib/modules/* $NV_WORKSPACE/drive-linux/kernel/preempt_rt${PROD_SUFFIX}/modules/
      sudo cp -a ${PWD}/out-linux-mlnx-build/mlnx-drivers-build/debian/mlnx-ofed-kernel-modules/lib/modules/* $NV_WORKSPACE/drive-linux/kernel/preempt_rt${PROD_SUFFIX}/modules/
    3. Copy the built modules to the root file system path with the following commands:
      1. To copy the updated modules into the root file system, use the following Build-FS steps:
        1. Create a Build-FS JSON with the following content using the editor of your choice. Name it update_rfs.CONFIG.json and put it in $PWD. The parameter <fstype> should be standard, production, production_debug depending on the list of modules to be copied to the filesystem.
          {
              "OS": "linux",
              "Output": "driveos-updated-rfs",
              "Base": "${BASE_DIR}/targetfs.img",
              "FilesystemType": "<fstype>",
              "CopyTargets": [
                  "${COPYTARGETYAML_DIR}/copytarget-kernel-modules.yaml"
              ],
              "PostInstalls": {
                  "/etc/nvidia/run-once/nv-run-once-run-ldconfig": "target",
                  "/etc/nvidia/run-once/nv-run-once-run-depmod": "target"
              }
          }
        2. Execute Build-FS with the preceding configuration to rebuild the filesystem. Set NV_WORKSPACE to opt for SDK install directory (containing drive-linux directory).
          export NVRTKERNELNAME="$(basename $NV_WORKSPACE/drive-linux/kernel/preempt_rt${PROD_SUFFIX}/modules/*rt*-tegra)"
          sudo -E /usr/bin/python3 -B /opt/nvidia/driveos/common/filesystems/build-fs/17/bin/build_fs.py -w ${NV_WORKSPACE}/ -i $PWD/update_rfs.CONFIG.json -o ${NV_WORKSPACE}/drive-linux/filesystem/targetfs-images/
          sudo rm -f ${NV_WORKSPACE}/drive-linux/filesystem/targetfs.img
          sudo ln -s ${NV_WORKSPACE}/drive-linux/filesystem/targetfs-images/driveos-updated-rfs.img ${NV_WORKSPACE}/drive-linux/filesystem/targetfs.img
  13. Update initramfs kernel modules with the ones built.
    1. If Building a production kernel, then please set
      1. export INITRAMFS_IMAGE=$NV_WORKSPACE/drive-linux/filesystem/prod-initramfs.cpio
    2. Else If building a standard kernel, please set
      1. export INITRAMFS_IMAGE=$NV_WORKSPACE/drive-linux/filesystem/initramfs.cpio
    3. Extract the initramfs.
      1. sudo rm -rf ./initramfs; sudo mkdir -p ./initramfs; cd ./initramfs
      2. sudo cpio -imdu --quiet < ${INITRAMFS_IMAGE}
    4. Run copytarget to copy kernel modules.
          export PROD_SUFFIX=<str> # If using production kernel, please set <str> to "_prod" else set to empty string ""
          export NVRTKERNELNAME="$(basename $NV_WORKSPACE/drive-linux/kernel/preempt_rt${PROD_SUFFIX}/modules/*rt*-tegra)"
          export NV_SDK_NAME_LINUX=drive-linux
          sudo rm -rf lib/modules/*
          sudo -E /opt/nvidia/driveos/common/filesystems/copytarget/1/copytarget.py $PWD $NV_WORKSPACE/ $NV_WORKSPACE/drive-linux/filesystem/copytarget/manifest/copytarget-kernel-modules.yaml --filesystem-type boot_initramfs
    5. Re-create updated initramfs:
      1. sudo find . | sudo cpio --quiet -H newc -o > ${INITRAMFS_IMAGE}
  14. Flash the target using the SDK Bootburn tool to the board.
  15. Boot up the board where the built kernel modules are auto-loaded on boot.