Blog de Joe Freeman

I struggled initially to manage my ML and data science projects when running them in the NVIDIA Workbench. The built-in GIT interface was basic  and fine for basic operations but I found it frustrating for branching and merging.  I ended up using tools that ran inside the Workbench Container instead of the Workbench itself. Workbench SVC support continues to improve but I use the other tools discussed while watching Workbench improve. Here are the 4 ways I've explored. The AI Workbench UI This NVIDIA AI Workbench interface is a work in progress.  The latest version adds git diff  capabilities, a big improvement.  This UI is still simple and intended for basic file commits.  Use something else if you need advanced git control or merge functionality. Jupyter Notebook Extensions There are a few different Jupyter Notebook extensions. Some, like jupyterlab-git,   add themselves to the Notebook UI toolbar. Here is the left gutter interface for extension:...

This is about using a Turing NVIDIA Titan RTX GPU to locally execute code assist LLMs to be used in VSCode. This slightly older card has 24GB of VRAM making it a great local LLM. The Titan RTX is a two-slot dual-fan card. The Titan RTX is currently about the same price as a refurbished Ampere NVIDIA 3090 TI 24GB. There are a bunch of ways to host the code support LLMs. We are using an early release  Ollama   as our LLM service and  continue.dev  VSCode extension as the language service inside VSCode.   This was tested on AMD Ryzen 8 core with 64GB of memory and the Titan RTX.  Related blog articles and videos Several related blogs and videos that cover VSCode and local LLMs Blog  Get AI code assist VSCode with local LLMs using Ollama and the Continue.dev extension - Mac Get AI code assist VSCode with local LLMs using LM Studio and the Continue.dev extension - Windows Rocking an older Titan RTX 24GB as my local AI Code assist...

This is about running VSCode AI code assist locally replacing Copilot or some other service.  You may run local models to guarantee none of your code ends up on external servers. Or, you may not want to maintain an ongoing AI subscription. We are going to use  LM  Studio and VS Code.  This was tested on Windows 11 with an RTX 3060 TI with 8GB of VRAM. 8GB really limits the number and size of the models we can use. LM Studio's simple hosting model of 1 LLM and an embedding works for us in this situation.  You want a big card. 8GB is a tiny card. Related blog articles and videos Several related blogs and videos that cover VSCode and local LLMs Blog  Get AI code assist VSCode with local LLMs using Ollama and the Continue.dev extension - Mac Get AI code assist VSCode with local LLMs using LM Studio and the Continue.dev extension - Windows Rocking an older Titan RTX 24GB as my local AI Code assist on Windows 11, Ollama and VS Code YouTube Video  Using loc...

This is about running VSCode AI code assist locally as a replacement for Copilot or some other service.  You may run local models to guarantee none of your code ends up on external servers. Or, you may not want to maintain an ongoing AI subscription. We are going to use Ollama  as our LLM service and the continue.dev  VSCode extension as the language service inside VSCode.   This was tested on a MacBook using the Apple GPU. Macs are an interesting platform for running local AI code assist and LLMs because you can treat much of main memory as GPU VRAM. Related blog articles and videos Several related blogs and videos that cover VSCode and local LLMs Blog  Get AI code assist VSCode with local LLMs using Ollama and the Continue.dev extension - Mac Get AI code assist VSCode with local LLMs using LM Studio and the Continue.dev extension - Windows Rocking an older Titan RTX 24GB as my local AI Code assist on Windows 11, Ollama and VS Code YouTube Video ...