Blog de Joe Freeman

This is about using a Turing NVIDIA Titan RTX GPU to locally execute code assist LLMs to be used in VSCode. This slightly older card has 24GB of VRAM making it a great local LLM. The Titan RTX is a two-slot dual-fan card. The Titan RTX is currently about the same price as a refurbished Ampere NVIDIA 3090 TI 24GB. There are a bunch of ways to host the code support LLMs. We are using an early release  Ollama   as our LLM service and  continue.dev  VSCode extension as the language service inside VSCode.   This was tested on AMD Ryzen 8 core with 64GB of memory and the Titan RTX.  Related blog articles and videos Several related blogs and videos that cover VSCode and local LLMs Blog  Get AI code assist VSCode with local LLMs using Ollama and the Continue.dev extension - Mac Get AI code assist VSCode with local LLMs using LM Studio and the Continue.dev extension - Windows Rocking an older Titan RTX 24GB as my local AI Code assist...

This is about running VSCode AI code assist locally as a replacement for Copilot or some other service.  You may run local models to guarantee none of your code ends up on external servers. Or, you may not want to maintain an ongoing AI subscription. We are going to use Ollama  as our LLM service and the continue.dev  VSCode extension as the language service inside VSCode.   This was tested on a MacBook using the Apple GPU. Macs are an interesting platform for running local AI code assist and LLMs because you can treat much of main memory as GPU VRAM. Related blog articles and videos Several related blogs and videos that cover VSCode and local LLMs Blog  Get AI code assist VSCode with local LLMs using Ollama and the Continue.dev extension - Mac Get AI code assist VSCode with local LLMs using LM Studio and the Continue.dev extension - Windows Rocking an older Titan RTX 24GB as my local AI Code assist on Windows 11, Ollama and VS Code YouTube Video ...