Manually validating compatibility and running NVIDIA (NIM) container images
NVIDIA NIMs are ready to run pre-packaged containerized models. The NIMs and their included models are available in a variety of profiles supporting different compute hardware configurations. You can run the NIMs in an interrogatory mode that will tell you which models are compatible with your GPU hardware. You can then run the NIM with the associated profile. Sometimes there are still problems and we have to add additional tuning parameters to fit in memory or change data types. In my case, the data type change is because of some bug in the NIM startup detection code. This article requires additional polish. It has more than a few rough edges. NVIDIA NIMs are semi-opaque. You cannot build your own NIM. NIM construction details are not described by NVIDIA. Examining NVIDIA Model Container Images The first step is to select models we think can fit and run on our NVIDIA GPU hardware. The first step is to investigate models ...