Logging
DRIVE OS has a logging component that allows logs from multiple DRIVE OS components to be stored (or) streamed out of DRIVE AGX hardware. Currently, support is only available for applications within DRIVE OS Linux Guest VM to send logs that can be stored in a file that can be created and is accessible from Linux Guest VM. This support will be enhanced in future releases to add more execution environments to generated logs that can be stored or streamed out of DRIVE AGX hardware.
Components
The logging pipeline is as follows :
Application(s) trying to log (referred to as Logging Entity in this document) -> libnvlog.so (logging library that is linked to Logging Entity) -> Logging Client (nvlogclient_fast) -> user configured storage file.
-
/usr/local/bin
nvlogclient_fast
: Application component responsible for collecting logs from different Logging Entities within Linux Guest VM and storing them.nvlogtest
: Test application that has two threads sending log messages every second.nvlogtest_stdin
: Test application that takes input from stdin and sends each line as a log message. /usr/local/lib/
libnvlog.so
: Logging library that provides logging APIs.
Source and Binary Locations
- Pre-build binaries:
<PDK_TOP>/ drive-linux/lib-target/libnvlog.so <PDK_TOP>/ drive-linux/filesystem/contents/bin/nvlogclient_fast <PDK_TOP>/ drive-linux/filesystem/contents/bin/nvlogtest <PDK_TOP>/ drive-linux/filesystem/contents/bin/nvlogtest_stdin
- Source code:
Source files for the test applications (nvlogtest*) and the Logging Client (nvlogclient_fast) can be found under :
<PDK_TOP>/ drive-linux/samples/tools/nvlog/.
- Include files and user API:
Any application needing to send logs using this new framework must use the
<PDK_TOP>/drive-linux/samples/tools/nvlog/include/nvlog.h and link to libnvlog.so
header file. API documentation can be found in the same header file.
Execution Steps
- Logging client needs to be started on each fresh boot:
nvlogclient_fast &
Note: nvlogclient_fast stores the output log in the current working directory asoutput_recv.txt
by default. This can be changed by passingnvlogclient_fast <log_file_full_path_and_name>
to the Logging Client application. -
nvlogtest & tail -f -n +1 /var/log/syslog | nvlogtest_stdin > output_sender.txt &
Note: nvlogtest and nvlogtest_stdin are two test applications that log test messages using the new logging framework.- nvlogtest has two threads. The main thread sends the
following log message every second :
"Hello World <sequence number> from main"
. The secondary thread sends the following log message every second:"Hello World <sequence number> from thread"
. - nvlogtest_stdin is a single threaded application that takes
stdin as input and sends each line as a log message. In the
above example,
/var/log/syslog
is read by nvlogtest_stdin and each line is sent as a log message.
- nvlogtest has two threads. The main thread sends the
following log message every second :
-
sleep 20 pkill nvlog
The commands above allow logging components to operate for 20 seconds and then kills
them. Output from nvlogclient_fast can now be read from the
output_recv.txt
file in the current working directory.
Sample Output
sort -k4 -n output_recv.txt > output_recv_sorted.txt
cat output_recv_sorted.txt
0 0 1 1656242248580689 266 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] Linux version 5.10.104-rt63-tegra (buildbrain@mobile-u64-5385-d8000) (aarch64-buildroot-linux-gnu-gcc.br_real (Buildroot 2020.08) 9.3.0, GNU ld (GNU Binutils) 2.33.1) #1 SMP PREEMPT RT Tue Jun 14 10:46:31 PDT 2022
0 0 2 1656242248580690 108 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] OF: fdt: memory scan node memory@80000000, reg size 32,
0 0 3 1656242248580691 85 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] OF: fdt: - 80000000 , 80000000
0 0 4 1656242248580691 87 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] OF: fdt: - 100000000 , 70d800000
0 0 5 1656242248580692 78 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] Machine model: p3710-0010
0 0 6 1656242248580692 73 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] efi: UEFI not found.
0 0 7 1656242248580693 141 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] OF: reserved mem: initialized node camdbg_carveout, compatible id nvidia,camdbg_carveout
0 0 8 1656242248580694 129 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] Reserved memory: created CMA memory pool at 0x00000007fd800000, size 256 MiB
0 0 10 1656242248580695 65 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] Zone ranges:
0 0 9 1656242248580695 128 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
0 0 11 1656242248580696 107 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] DMA [mem 0x0000000080000000-0x00000000ffffffff]
0 0 12 1656242248580696 69 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] DMA32 empty
0 0 13 1656242248580697 107 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] Normal [mem 0x0000000100000000-0x000000080d7fffff]
0 0 14 1656242248580697 85 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] Movable zone start for each node
0 0 15 1656242248580698 77 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] Early memory node ranges
0 0 16 1656242248580698 108 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] node 0: [mem 0x0000000080000000-0x00000000dd33ffff]
0 0 17 1656242248580699 108 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] node 0: [mem 0x00000000dd340000-0x00000000dd37ffff]
0 0 18 1656242248580699 108 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] node 0: [mem 0x00000000dd380000-0x0000000199bfffff]
0 0 19 1656242248580700 108 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] node 0: [mem 0x00000001f9c00000-0x000000080d7fffff]
0 0 20 1656242248580700 117 - Jun 15 05:30:49 tegra-ubuntu kernel: [ 0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x000000080d7fffff]
....
0 0 1312 1656242248628536 88 - Jun 15 05:30:49 tegra-ubuntu systemd-networkd[1317]: mgbe3_0: IPv6 successfully enabled
0 0 1313 1656242248628537 88 - Jun 15 05:30:49 tegra-ubuntu systemd-networkd[1317]: mgbe2_0: IPv6 successfully enabled
0 0 1314 1656242248628538 88 - Jun 15 05:30:49 tegra-ubuntu systemd-networkd[1317]: mgbe1_0: IPv6 successfully enabled
0 0 1315 1656242248628538 88 - Jun 15 05:30:49 tegra-ubuntu systemd-networkd[1317]: mgbe0_0: IPv6 successfully enabled
0 0 1316 1656242248628539 87 - Jun 15 05:30:49 tegra-ubuntu systemd-networkd[1317]: eqos_0: IPv6 successfully enabled
0 0 1317 1656242248628539 104 - Jun 15 05:30:50 tegra-ubuntu kernel: [ 149.652480] 8021q: adding VLAN 0 to HW filter on device mgbe3_0
0 0 1318 1656242248628540 133 - Jun 15 05:30:50 tegra-ubuntu kernel: [ 149.655826] nvethernet 6b10000.ethernet mgbe3_0: Link is Up - 10Gbps/Full - flow control off
0 0 1319 1656242248628541 111 - Jun 15 05:30:50 tegra-ubuntu kernel: [ 149.665770] hwmon hwmon1: temp1_input not attached to any thermal zone
0 0 2 1656242248629107 Hello World 1 from main
0 0 3 1656242248629145 Hello World 1 from thread
0 0 1320 1656242248629609 104 - Jun 15 05:30:50 tegra-ubuntu kernel: [ 149.706221] 8021q: adding VLAN 0 to HW filter on device mgbe1_0
0 0 1321 1656242248629612 120 - Jun 15 05:30:50 tegra-ubuntu kernel: [ 149.706370] Aquantia AQR113C 6910000.ethernet:00: No AQR phy_mode setting in DT
0 0 1322 1656242248629613 77 - Jun 15 05:30:50 tegra-ubuntu systemd-resolved[1383]: Positive Trust Anchors:
...
0 0 98 1656242296632042 Hello World 49 from thread
0 0 99 1656242296632068 Hello World 49 from main
0 0 100 1656242297632090 Hello World 50 from thread
0 0 101 1656242297632135 Hello World 50 from main
0 0 102 1656242298632127 Hello World 51 from thread
0 0 103 1656242298632199 Hello World 51 from main
0 0 104 1656242299632158 Hello World 52 from thread
0 0 105 1656242299632244 Hello World 52 from main
0 0 106 1656242300632222 Hello World 53 from thread
0 0 107 1656242300632308 Hello World 53 from main
0 0 108 1656242301632286 Hello World 54 from thread
0 0 109 1656242301632373 Hello World 54 from main
0 0 110 1656242302632349 Hello World 55 from thread
0 0 111 1656242302632439 Hello World 55 from main
0 0 112 1656242303632365 Hello World 56 from thread
0 0 113 1656242303632504 Hello World 56 from main
<End of File>