Blog de Joe Freeman

Data scientists and others want to analyze and manipulate lots of data without having to be Software developers. Jupyter notebooks, Python and no DASK are vehicles that help make that happen. DASK is a distributed Python library that can run in pretty much any Python environment.  It was designed to help data scientists scale up their data analysis by providing an easy to use distributed compute API and paradigm. DASK/Python can be run both inside Jupyter notebooks and as standalone Python programs. Environment The DASK environment consists of a Dask Scheduler and any number of worker nodes. The Python program sends a set of tasks to the Dask Scheduler which then distributes those across the worker nodes.  The worker node results are then aggregated  and returned to the original program. Docker The DASK team provides samples how to use docker-compose to create a DASK development and execution environment The environment consists of A development no...

Azure Sphere is another example of  how vendors can create great end to end solutions. Time will tell if it will become another cautionary tale about vendor lock in and corporate attention spans. Why Azure Sphere? Azure Sphere is an end-to-end IOT platform to build applications, deploy them, manage them, securely communicate with them and and remotely update them. Microsoft spent the last few years learning the complexity of building and running secure IOT environments. The continual stream of IOT device hacks and attacks shows that building and running secure, scale-able IOT clouds is beyond the capabilities of many companies. Azure Sphere simplifies this by pushing some of the responsibilities onto Microsoft itself. Azure  Microsoft's Software Effort  Microsoft is putting in a real effort create a professional grade  IOT ecosystem with their Azure Sphere project. They have built a hardware and software platform where security and cloud integration ...

This post explains how Hyper-V and Docker reserve memory on Windows machines that is outside of the normal process allocations.  It shows how you can see those reservations. Docker and Hyper-V Docker Desktop on Windows runs inside a Hyper-V virtual machine.  The virtual machine's memory is sized specified on the Memory  setting in the Docker Control panel's Advanced  tab. Docker Desktop memory reservation. That memory sizing maps directly to the Docker virtual machine's  Assigned Memory value that is displayed in the Hyper V Manager  Virtual Machines view. Hyper-V locks the virtual machine into memory by Drover Locking  the VM's memory.  Memory used for VMs is partitioned away from the rest of the host's processes. Hyper-V Manager VM's view Hyper-V guest memory does not appear in TaskManager  because it is reserved outside of the normal process structures. Memory allocation by type with RamMap Hyper-V virtual machine me...

I've been struggling with my Dell laptop running Windows 10.  The machine seems to shutdown overnight or after a few hours even when it's plugged in. It turns out my machine is hibernating which doesn't actually work right on my machine.  My temporary fix is to extend the  Hibernate after  value to 5 days, 7200 minutes I used this forum posting  as a guide. Default  Power Options values for Balanced  profile Steps Press the Windows Start key or the  Start menu  button  Enter "power and sleep" and press Enter Click on Additional power settings Click on Change plan settings for the selected plan Click on Change when the Computer sleeps Click Change Advanced Settings Expand Sleep  (click on +) Expand Hibernate after Change the  Plugged in:  value to  7200 min Click the  Apply  button

Do you want to try out new technologies without installing a bunch of software inside your local machine?  Do you wish to edit, build and debug code on commonly available Docker containers without having to build custom development containers? Visual Studio Code with  remote extensions  let you do just that. I wanted to play with Machine Learning, Jupyter Notebooks, and TensorFlow without mucking up my local machine. ML tooling is large and complicated to install. The easiest way to do this is to run Anaconda/Jupyter/tensor-flow in containers derived from unadulterated  publicly available  Docker images. Jupyter notebooks are great but sometimes I want an IDE. VS Code Remote  lets me use a standard IDE while developing in a remote sandbox. It provides command line access without forcing me to command line-based tooling. Visual Studio Code can build, run and debug on a remote machine via its various remote extensions .  Remote Extension for Docke...