Optimizing Boot KPI#

This chapter lists changes to improve Boot KPI in production builds. The exact definition of Boot KPI is provided by each customer. This section also calls out the method to demonstrate end-to-end Boot KPI measurement involving single Rear View Camera (RVC) + Display or Surround View System (SVS) with 4 Cameras and Display.

Improve Boot KPI#

To improve Boot KPI, 3 additional changes have to be applied on Top of SDK. These changes are only needed if you are not using “prod_nsr” PCT variant.

  1. Disabling Foundation Logs

    1. Disable MB1_MB2 logs

      1. Open DT file ${NV_WORKSPACE}/drive-foundation/platform-config/hardware/nvidia/platform/t264/common/bct/mb1bct/debug/tegra264-mb1-bct-debug-defaults.dtsi

      2. Change log_level property to <0> under debug DT node.

    Disabling BPMP Logs

    1. Open the BPMP UART file ${NV_WORKSPACE}/drive-foundation/./hardware/nvidia/platform/t264/bpmp-dtb/include/policy/utc-enable.dtsi

    2. In the above file, remove the contents of the “serial” node. For example:

    Before

    / {
    serial {
        port = <3>;
        has_input;
        combined-uart {
            enabled;
        };
    };
};

    After

    / {
    serial {
    };
};

  1. Disable DEBUGFS in Linux Kernel - Pass debugfs=off as part of the kernel command line. This method allows you to keep CONFIG_DEBUG_FS enabled but still disables it using the kernel command line argument.

  2. Reduce Linux Kernel log level - Pass loglevel=3 as part of the kernel command line. This sets the Linux Kernel log level to only print errors.

  3. Disable PCIe Port C2 if no NVMe device is connected on the board - Add ‘’ENABLE_T264_PCIE_C2=n’’ to bind command arguments to disable C2 if there is no NVMe device connected to the board. This will prevent a delay of 1.2 delay when PCIe Server is initialized.

Demonstrate end-to-end Boot KPI measurement#

Invoking “init” Script#

Boot KPI measurements take place right after initramfs. Usually, once initramfs completes, the “init” program is launched. In this case, “init” is just a symbolic link to “systemd”. So to measure boot KPIs, you can instead point the “init” executed by the system to a script that runs your Boot KPI program, then launches systemd. A script to do this has been created and is present by default on the system: /usr/bin/nv_camera_display.sh This script loads all necessary kernel modules and launches the sample RVC / SVS application, then launches systemd. Add init=/usr/bin/nv_camera_display.sh to the kernel command line. After doing changes for improving Boot KPI and invoking the init script, do Bind, Create BSP images and Flash operations.

Note

Kernel command line can be updated by following these steps:

  1. Open ${NV_WORKSPACE}/drive-foundation/platform-config/hardware/nvidia/platform/t264/automotive/pct/drive_av/linux/linux_storage.cfg

  2. Add debugs=off loglevel=3 init=/usr/bin/nv_camera_display.sh to os_args

  3. This can be also done setting flag ENABLE_BOOT_KPI_ARGS to y in ${NV_WORKSPACE}/drive-foundation/platform-config/hardware/nvidia/platform/t264/automotive/pct/drive_av/linux/common_profile.mk

Update nv_camera_display.sh#

Please make the following change in nv_camera_display.sh.

Note

This change will not be needed in 7.0.3.0 release, as these change will be the default.

In the following example, edits are indicated with the following:

➕ Added line

➖ Deleted line

   /sbin/modprobe tegra_bootloader_debug

  ➖echo -n "Load LKMs Start" > /sys/kernel/tegra_bootloader/add_profiler_record
  ➖/sbin/modprobe -a cdi_mgr cam_fsync nvmap nvgpu host1x_fence \
  ➖    nvhost-capture nvhost-isp5 nvhost-nvcsi nvhost-nvcsi-t194 nvhost-vi5 tegra-drm nvdisp_serdes
  ➕SOC_ID=$(cat /sys/devices/soc0/soc_id)

  ➖echo -n "Load Display LKMs Start" > /sys/kernel/tegra_bootloader/add_profiler_record
  ➖/sbin/modprobe nvidia rm_firmware_active=all
  ➖/sbin/modprobe nvidia-modeset
  ➕if [ "$SOC_ID" = "38" ]; then
  ➕    echo -n "Load LKMs Start" > /sys/kernel/tegra_bootloader/add_profiler_record
  ➕    /sbin/modprobe -a cdi_mgr cam_fsync nvmap host1x_fence \
  ➕        nvhost-capture nvhost-isp5 nvhost-nvcsi nvhost-nvcsi-t194 nvhost-vi5 nvgpu \
  ➕        tegra-drm nvdisp_serdes
  ➕    echo -n "Load New Display Start" > /sys/kernel/tegra_bootloader/add_profiler_record
  ➕    /sbin/modprobe tegradisp
  ➕elif [ "$SOC_ID" = "35" ]; then
  ➕    echo -n "Load LKMs Start" > /sys/kernel/tegra_bootloader/add_profiler_record
  ➕    /sbin/modprobe -a cdi_mgr cam_fsync nvmap nvgpu host1x_fence \
  ➕       nvhost-capture nvhost-isp5 nvhost-nvcsi nvhost-nvcsi-t194 nvhost-vi5 \
  ➕        tegra-drm nvdisp_serdes
  ➕
  ➕   echo -n "Load Display LKMs Start" > /sys/kernel/tegra_bootloader/add_profiler_record
  ➕   /sbin/modprobe nvidia rm_firmware_active=all
  ➕    /sbin/modprobe nvidia-modeset
  ➕else
  ➕    echo "Unknown SOC_ID: $SOC_ID"
  ➕fi

echo -n "Launching app" > /sys/kernel/tegra_bootloader/add_profiler_record

Viewing Profiling Results#

In the process of running /usr/bin/nv_camera_display.sh, timestamps for relevant events are recorded into a dedicated memory region.

These timestamps can be viewed after boot as follows:

  1. Ensure the tegra_bootloader_debug.ko kernel module is loaded. It is loaded as part of nv_camera_display.sh, but in case that was not run, it can be loaded from /lib/modules/$(uname -r)/kernel/drivers/platform/tegra/tegra_bootloader_debug.ko.

  2. Expose the timestamps in the memory region:

    sudo cat /sys/kernel/tegra_bootloader/profiler

  • The timestamps are now present in /var/log/syslog. They are visible at the end of the file.

  1. For example, using the less program, open the file less /var/log/syslog.

    • Press the “End” key on your keyboard.

    Example profiling log:

root@tegra-ubuntu:/home/nvidia#  sudo cat /sys/kernel/tegra_bootloader/profiler
MB1 start                                                          77147
MB1 finish                                                        300510                  223363

MB2 Entry                                                         309811
MB2 Start                                                         331792                   21981
MB2 Device Init Done                                              386127                   54335
MB2 Finished                                                      908178                  522051

HV: Start                                                        1061786
HV: Start loading containers                                     1063017                    1231
HV: Done loading containers                                      1063875                     858
HV: Done with guest_init                                         1066974                    3099
HV: Start memory allocation & mapping(ivc, guest)                1067193                     219
HV: Done with memory operations                                  1088573                   21380
HV: Before printing hello                                        1088580                       7
HV: Start Asynchronous SMP                                       1088665                      85

pl: EL1 Init TS                                                  1228943
pl: LoadRamdisk                                                  1278231                   49288
pl: LoadDTB                                                      1286948                    8717
pl: LoadKernel                                                   1327428                   40480
pl: Kernel Handoff                                               1327430                       2
Load LKMs Start                                                  2029606                  702176
Load New Display Start                                           2364783                  335177
Launching app                                                    2466923                  102140
nvsipl_multicast: main                                           2524635                   57712
nvsipl_multicast: First frame displayed                          4485464                 1960829
  1. How to interpret the data:

    There are two columns with relevant numbers. a. The first column (second from right) is the absolute time since boot it took for the associated event to take place in microseconds (usec). b. The second column (first from right) is the delta between the associated event and the previous event in usec. For example, the “Launching app” event took place 3809333 usec (3.8 sec) after boot and occurred 104326 usec (104 milliseconds) after the previous event, insmod nvidia.ko.

  2. Use your RVC/SVS application: You can invoke your RVC/SVS application in place of the existing application to validate the Boot KPI on your platform.

On this page