Are you in the UK? Ultrafast Internet up to 8Gbps. Switch to YouFibre
Are you in the UK? Ultrafast Internet up to 8Gbps. Switch to YouFibre

Running Docker Containers Directly on NVIDIA DRIVE AGX Orin

GIXnews

NVIDIA DRIVE OS host-side Docker containers on NGC provide a quick and easy alternative to installing Debian packages to set up NVIDIA DRIVE OS on a host…

NVIDIA DRIVE OS host-side Docker containers on NGC provide a quick and easy alternative to installing Debian packages to set up NVIDIA DRIVE OS on a host development machine. 

Developers using the NVIDIA DRIVE platform have requested the ability to run Docker containers directly on NVIDIA DRIVE AGX hardware for quick environment setup to run applications with many package dependencies. 

NVIDIA DRIVE OS 6.0.6 Linux now includes the NVIDIA Container Toolkit runtime as part of the target Root File System (RFS) that is flashed onto the board. It enables developers to run Docker containers directly on the target NVIDIA DRIVE AGX hardware. 

This post explains how to run a few sample applications inside a Docker container running on the target NVIDIA DRIVE AGX hardware.

Setup 

Install NVIDIA DRIVE OS on the host machine and flash the target board. See the NVIDIA DRIVE OS 6.0 Installation Guide for details.

After flashing NVIDIA DRIVE AGX Orin with NVIDIA DRIVE OS 6.0.6, you can pull target-side Docker images from NGC or Docker Hub, and run GPU-accelerated containers on the target right out of the box.

Running CUDA sample inside target-side Docker container 

The RFS flashed onto the target hardware using NVIDIA DRIVE OS 6.0.6 provides CUDA samples at /usr/local/cuda-11.4/samples. The matrix multiplication CUDA sample runs inside an Ubuntu 20.04 Docker container to demonstrate the accessibility of the GPU on NVIDIA DRIVE AGX Orin inside the container. 

$ cd /usr/local/cuda-11.4/samples/0_Simple/matrixMul && sudo make
$ sudo docker run --rm --runtime nvidia --gpus all -v $(pwd):$(pwd) -w $(pwd) ubuntu:20.04 ./matrixMul

The following output shows the successful running of the CUDA sample:

nvidia@tegra-ubuntu:/usr/local/cuda-11.4/samples/0_Simple/matrixMul$ sudo docker run --rm --runtime nvidia --gpus all -v $(pwd):$(pwd) -w $(pwd) ubuntu:20.04 ./matrixMul
[sudo] password for nvidia: 
[Matrix Multiply Using CUDA] - Starting...
GPU Device 0: "Ampere" with compute capability 8.7

MatrixA(320,320), MatrixB(640,320)
Computing result using CUDA Kernel...
done
Performance= 617.91 GFlop/s, Time= 0.212 msec, Size= 131072000 Ops, WorkgroupSize= 1024 threads/block
Checking computed result for correctness: Result = PASS

NOTE: The CUDA samples are not meant for performance measurements. Results may vary when GPU Boost is enabled.

Running custom applications inside a target-side Docker container 

Two files—devices.csv and drivers.csv—are provided within the RFS flashed onto the board for applications inside Docker containers to access the devices, drivers, and shared libraries. These files each have a list of devices, drivers, and shared libraries needed for an application to run successfully inside the Docker container. 

These files must be edited to reflect all the devices, drivers, and shared libraries the application needs to run custom applications inside the Docker containers. To access these two files, see /etc/nvidia-container-runtime/host-files-for-container.d/.

For example, the CUDA sample cudaNvSci does not run out of the box, unlike the MatrixMul sample. An attempt to run the CUDA sample inside the Docker container without any changes to drivers.csv results in the following error:

nvidia@tegra-ubuntu:cd /usr/local/cuda-11.4/samples/0_Simple/cudaNvSci && sudo make
nvidia@tegra-ubuntu:/usr/local/cuda-11.4/samples/0_Simple/cudaNvSci$ sudo docker run --rm --runtime nvidia --gpus all -v $(pwd):$(pwd) -w $(pwd) ubuntu:20.04 ./cudaNvSci
./cudaNvSci: error while loading shared libraries: libnvscibuf.so.1: cannot open shared object file: No such file or directory

The CUDA sample does work outside the Docker container. The output of running the sample directly on the target is shown below:

nvidia@tegra-ubuntu:/usr/local/cuda-11.4/samples/0_Simple/cudaNvSci$ ./cudaNvSci
1 GPUs found
Loaded './lenaRGB.ppm', 1024 x 1024 pixels
[cudaNvSciSignal] GPU Device 0: "Ampere" with compute capability 8.7
 
[cudaNvSciWait] GPU Device 0: "Ampere" with compute capability 8.7
 
created NvSciBufObj
created NvSciBufImageObj
created NvSciSyncObj
Wrote 'lenaRGB_out.ppm'

The output (above) should result if the sample is run inside the Docker container. Instead, an error results from loading the shared library libnvscibuf.so.1. To fix this error, look at the contents of drivers.csv:

dir, /usr/lib/firmware/tegra23x
 
lib, /usr/lib/libcuda.so.1
lib, /usr/lib/libnvrm_gpu.so
lib, /usr/lib/libnvrm_mem.so
lib, /usr/lib/libnvrm_sync.so
lib, /usr/lib/libnvrm_host1x.so
lib, /usr/lib/libnvos.so
lib, /usr/lib/libnvsocsys.so
lib, /usr/lib/libnvtegrahv.so
lib, /usr/lib/libnvsciipc.so
lib, /usr/lib/libnvrm_chip.so
lib, /usr/lib/libnvcucompat.so

This list does not include libnvscibuf.so.1 that the sample requires to run inside the Docker container. 

At this point, tools like ldd and objdump can help to determine the shared libraries required by the sample so the drivers.csv file can be edited to include them. 

Running ldd on cudaNvSci results in the following output: 

nvidia@tegra-ubuntu:/usr/local/cuda-11.4/samples/0_Simple/cudaNvSci$ ldd ./cudaNvSci
    	linux-vdso.so.1 (0x0000ffffa9f2b000)
    	libcuda.so.1 => /usr/lib/libcuda.so.1 (0x0000ffffa8933000)
    	libnvscibuf.so.1 => /usr/lib/libnvscibuf.so.1 (0x0000ffffa88c1000)
    	libnvscisync.so.1 => /usr/lib/libnvscisync.so.1 (0x0000ffffa888f000)
    	librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000ffffa8877000)
    	libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000ffffa8846000)
    	libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000ffffa8832000)
    	libstdc++.so.6 => /lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000ffffa864d000)
    	libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000ffffa8629000)
    	libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffffa84b6000)
    	/lib/ld-linux-aarch64.so.1 (0x0000ffffa9efb000)
    	libnvrm_gpu.so => /usr/lib/libnvrm_gpu.so (0x0000ffffa844a000)
    	libnvrm_mem.so => /usr/lib/libnvrm_mem.so (0x0000ffffa8431000)
    	libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffffa8386000)
    	libnvos.so => /usr/lib/libnvos.so (0x0000ffffa8365000)
    	libnvscicommon.so.1 => /usr/lib/libnvscicommon.so.1 (0x0000ffffa8350000)
    	libnvsciipc.so => /usr/lib/libnvsciipc.so (0x0000ffffa8325000)
    	libnvrm_host1x.so => /usr/lib/libnvrm_host1x.so (0x0000ffffa8303000)
    	libnvsocsys.so => /usr/lib/libnvsocsys.so (0x0000ffffa82ef000)
    	libnvtegrahv.so => /usr/lib/libnvtegrahv.so (0x0000ffffa82dc000)
    	libnvrm_sync.so => /usr/lib/libnvrm_sync.so (0x0000ffffa82c4000)
    	libnvrm_chip.so => /usr/lib/libnvrm_chip.so (0x0000ffffa82b0000)

This list shows all the shared libraries the cudaNvSci sample requires, including libnvscibuf.so.1. 

Now, edit drivers.csv to include the shared libraries that are not already present in the list. After adding the required libraries, the content of drivers.csv is as follows (with bolded lines newly added):

dir, /usr/lib/firmware/tegra23x
 
lib, /usr/lib/libcuda.so.1
lib, /usr/lib/libnvrm_gpu.so
lib, /usr/lib/libnvrm_mem.so
lib, /usr/lib/libnvrm_sync.so
lib, /usr/lib/libnvrm_host1x.so
lib, /usr/lib/libnvos.so
lib, /usr/lib/libnvsocsys.so
lib, /usr/lib/libnvtegrahv.so
lib, /usr/lib/libnvsciipc.so
lib, /usr/lib/libnvrm_chip.so
lib, /usr/lib/libnvcucompat.so
lib, /usr/lib/libnvscibuf.so.1
lib, /usr/lib/libnvscisync.so.1
lib, /lib/aarch64-linux-gnu/librt.so.1,
lib, /lib/aarch64-linux-gnu/libpthread.so.0
lib, /lib/aarch64-linux-gnu/libdl.so.2
lib, /lib/aarch64-linux-gnu/libstdc++.so.6
lib, /lib/aarch64-linux-gnu/libgcc_s.so.1
lib, /lib/aarch64-linux-gnu/libc.so.6
lib, /lib/aarch64-linux-gnu/libm.so.6,
lib, /usr/lib/libnvscicommon.so.1

After editing the drivers.csv file, the cudaNvSci sample is now able to run inside the Docker container:

nvidia@tegra-ubuntu:/usr/local/cuda-11.4/samples/0_Simple/cudaNvSci$ sudo docker run --rm --runtime nvidia --gpus all -v $(pwd):$(pwd) -w $(pwd) ubuntu:20.04 ./cudaNvSci
1 GPUs found
Loaded './lenaRGB.ppm', 1024 x 1024 pixels
[cudaNvSciSignal] GPU Device 0: "Ampere" with compute capability 8.7
 
[cudaNvSciWait] GPU Device 0: "Ampere" with compute capability 8.7
 
created NvSciBufObj
created NvSciBufImageObj
created NvSciSyncObj
Wrote 'lenaRGB_out.ppm'

Similarly, you can edit devices.csv and drivers.csv to add the devices and drivers required by custom applications to run them inside the Docker container on the target device. 

Conclusion

NVIDIA DRIVE OS 6.0.6 Linux now supports running Docker containers directly on the NVIDIA DRIVE AGX Orin hardware. Many samples should run inside the Docker container right after flashing, but some applications might require access to other devices and drivers, which can be accomplished by editing the devices.csv and drivers.csv files.

Source:: NVIDIA