Blog de Joe Freeman

Python has become the language of choice for data scientists and data analysts. It is easy to use with a lot analytical support libraries. Python programs aren't particularly fast. This has driven people to create a set of tools that help Python programs scale-up and scale-out.  We can compare approaches with a simple program that I adapted to JIT, GPU and distributed computing. Demonstration Caveat Distributed computing works best when there are problems with lots of I/O that can be spread across workers.  This program has no I/O and a high data transfer to computation ratio. GPUs have a high cost for data load and unload data with GPU memory. This means they work best when there is a significantly higher computation to data transfer ratio than this program has. Demonstration The sample Python program creates 10,000,00 3 variable rows that it then uses as inputs for 10,000,000 iterations of log(x)*log(y)*log(Z).  The results are returned in a 10,000,000 long...

You can run push portions of your Python programs to NVidia GPUs with Numba.  The install instructions are here .  Notes and Tips NVidia official GPU drivers are installed. Opensource GPU drivers don't work per Numba Installation guide CUDA development kit is not required per the Numba install instructions See Seven Things You Might Not Know About Numba Install CUDA Python Support Run Python Anaconda Install Anaconda Follow  Linux installation instructions  on Anaconda site Download the .sh script bash the .sh script source ~/.bashrc to add conda to the PATH of the current terminal Install Cuda Python and JIT conda install numba & conda install cudatoolkit Verify Python program Use the program at the bottom of this page  Sample Python Program See  https://github.com/freemansoft/python-playground  for sample programs Change log Created 2020 03

I wanted install the NVidia Geforce drivers and CUDA development environment on fresh Ubuntu 18.04 installation.  The target machine is an HP Z820 with a Geforce RTX 2060 super card. This assumes you want to write C code when building applicatoins for GPUs The Numba Installation Instructions say you only need the GPU drivers if you all your GPU programming is going to be done in Python. What didn't work Enabling proprietary drivers on installation did not work out of the box. The out of the box install left my login process broken. The screen would turn black and a cursor would appear and then would cycle back to the login screen. The problem was the display driver/windows system crashed and then restarted back at login.  I was able to fix it by uninstalling the driver or doing an upgrade. Assumptions NVIDIA drivers are not installed.  You should remove the drivers and reboot before proceeding if the drivers are loaded.  Instructions exist on the inter...

Chrome OS is not just for kids and coffee tables. It can provide a useful Linux platform too. Crostini brings a secure linux environment to Chrome OS.  Sophisticated development and deployments can be created with Docker containers running inside in this environment.  Chrome OS is capable of running Anaconda, Jupyter notebooks and other containers on many different developer machines. I run CloudReady Chrome OS on a Dell E7x40 to great effect.  References Chromebook firebase setup guide Cloud Ready running Docker without Crostini or VirtualBox Crostini forums on reddit Install docker-compse on Ubuntu Run Linux programs on ChromeBook with Crostini Hackernoon  Pixelbook Revisited: Running Docker Containers Running the shell and connecting Chrome to docker apps Install and run Docker in Crostini All testing was done on CloudReady running on a Dell E7240. I was able to run anaconda/Python and other docker-compose using docker-compose.yml files my...