Blog de Joe Freeman

Data science and AI projects are typically run on rented CPU/GPU resources or in self-managed data centers. By using containerization, we can separate our code and configuration from the underlying hardware, making it easier to switch between different systems for cost or performance reasons. However, containerization and remote execution introduce new networking and connectivity challenges. In this talk, we will explore how to manage connectivity for both the control plane and the user experience plane, including requirements like web access. AI transcription of the talk This section contains Gemini's rewrite of the YouTube Video's transcript. The raw text is down below.  The Role of Containerization in Local and Remote Data Science When designing a GPU-bound data exploration or analytics environment , the key to maintaining consistency—whether you're working locally or remotely —is a containerized workflow . For quick prototyping, we often leverage a few local GPUs witho...

The hot ticket right now is "Put AI Everywhere". This usually results in a race to come up with ideas about how AI can be used in new products or existing systems.  I suggest creating a system to analyze existing processes, software, and programs with a structured approach, identifying opportunities and assessing risks and rewards. LLM agents, chat bots and instructors models that turn human speech and writing patterns into content or actions is in scope here.  TTS, STT, translators and other non integrated LLM uses are out of scope. The talk below iteratively breaks down our process until we reach the point where we identify AI opportunities. We start with PDCA (Plan-Do-Check-Adjust) as a notion for the lifecycle of products, software, and processes. We can map that into the following in the software space. Item 4 could be anything specific to your domain.  Design Time: All of the processes that happen before actual code execution. This is before transactions, customer ...

Kaggle runs different machine learning or data science competitions. You can participate using their containerized environments or by coding locally. NVIDIA simplified working locally, or in your own cloud, with an AI Workbench-compatible  example Kaggle competition kernel . Their project contains everything needed to download competition data from Kaggle, run train/test cycles, and then upload the results for evaluation. I love this dockerized project because it lets me play in a competition sandbox on my local machine with no local configuration changes to my development machine The  Handwritten Digits Recognizer  competition is an open-ended competition trainer. Kaggle provides images of handwritten digits. You train against the training dataset and test against the testing dataset. Then run your trained model against the candidate digits of the competition submission set and upload the results to the Kaggle competition. The  NVIDIA Workbench example Kaggle c...

Some people buy computing power for self-training or self-edification projects. Others rent computing power. I like owning the gear I work on. I purchased a desktop to be used as a gaming machine, a development system, a containerized workload machine, a Data Science machine, and a Machine Learning platform. I didn't understand that it would turn into a series of hardware upgrades bound by PC architecture constraints. This was a great learning experience, but not the best raw dollars investment from a pure cost/capabilities point of view. The 3 years of upgrades cost $1200. I saved some money by purchasing previous-generation hardware. Buying current-generation hardware upgrades would cost $1900.  I could have stopped anywhere on the path. Apple wasn't a player in the gaming, M/L, or GPU market when I made my purchase. Apple has caught up for most of my use cases with its large shared memory architecture and performant CPUs. A MacBook might be the simplest approach for someone ...

NVidia AI Studio creates and manages containerized ML environments that isolate ML projects on local and remote machines.  You no longer have to switch environments  or remember which version of Python or Anaconda you are using in your global machine environment. NVIDIA simplifies the initial configuration by providing predefined image definitions containing Python, PyTorch, and other tools to be used with or without NVIDIA graphics cards. The actual development is done via browser-based tools like JupyterLab notebooks. Workbench spins up local proxies that port forward into the development container. See videos below NVIDIA Workbench runs in a WSL instance NVIDIA Workbench runs in its own WSL instance.  Each project runs in its own Docker container.  You can look at the NVIDIA main WSL instance by opening a shell into that WSL instance. The following command can be run in a Windows terminal window. wsl -d NVIDIA-Workbench NVIDIA projects live in the WSL instance in...