Use an NVIDIA Container Image to verify container access to NVIDIA GPUs
Verify that your container can communicate with the GPU and the type of NVidia GPUs your docker containers can access using one of NVIDIA's container images. docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi Sample Output The containers on this server have access to a single Titan RTX card Source This came from Nvidia getting started with large language models Revision Created 2024/07