Blog de Joe Freeman

I do a lot of my development and configuration via ssh into my Raspberry Pi Zero over the RNDIS connection. Some models of the Raspberry PIs can be configured with  gadget drivers  that let the Raspberry pi emulate different devices when plugged into computers via USB. My favorite gadget  is the network profile that makes a Raspberry Pi look like an RNDIS-attached network device.  All types of network services travel over an RNDIS device without knowing it is a USB hardware connection. A Raspberry Pi shows up as a Remote NDIS (RNDIS) device when you plug the Pi into a PC or Mac via a USB cable. The gadget  in the Windows Device Manager picture shows this RNDIS Gadget connectivity between a Windows machine and a Raspberry Pi. The Problem Windows 11 and Windows 10 no longer auto-installs the RNDIS driver that makes magic happen. Windows recognizes that the Raspberry Pi is some type of generic USB COM device. Manually running W indows Update  or  Upd...

Azure Application Insights is my go-to Observability platform when I want it to "just work". Application Insights can be a collection point for custom and third-party library Python Tracing and Transaction data. You can easily send Python Open Census span data from anywhere in the world into Azure Application Insights. You see your program transaction details from any browser that can reach the Azure console.  OpenCensus  spans are captured in  Application Insights  as  dependencies . The  OpenCensus Azure Exporter  supports nested spans letting you capture and measure nested operations. Every captured span is stored as a row in the Application Insights dependencies table.  The chart to the right was created from this sample code that has one main span and 4 nested sub-spans From FreemanSoft SpeedTest project https://github.com/freemansoft/speedtest-app-insights Executio...

Aligning expectations with partner capabilities. You may find yourself in a position where you are debating on whether bring on partners as part of an expansion strategy. The idea is to leverage your partners and grow your sales at a non-linear rate greater than your staff expansion rate. How partners Really Operate I've worked in a couple of places where we wanted to open up additional sales channels or bring on partners for moderately complex products.   We thought we could extend our reach by adding more, partners, and brains to the sales channel. You need to really dig in and understand if your partner is really going to be a  fulfillment vehicle  or a  demand-creation  engine. The partners could expand our visibility into areas we didn't have contact with. They always claimed that they could put us in front of their customers at a lower per-customer transaction price including their commission or fees.  It turns out the partners often would fall b...

Aggregating and analyzing streaming data is one of the ways people build machine learning datasets.  Data is ingested and then data near each other is pushed into aggregations or rows. Aggregations have several attributes or Features . You can think of them as columns in a database or spreadsheet. A data set is made up of many aggregations each one representing some subset of the stream data.  You can think of the aggregations as rows in a spreadsheet.  One of the challenges is picking the right windowing strategy for aggregating or analyzing streaming data. There are a variety of well-known windowing algorithms, Tumbling, Hoping, Sliding, etc. We are using a Tumbling Windows algorithm because of its relative simplicity and low memory usage. Tumbling windows repeat without overlap. Tumbling windows are either size-limited or time-limited. They contain a maximum amount of data or extend for a maximum amount of time. Time-based windowing: ...

Inter-system and Intra-system tracing capabilities are a must in modern distributed architectures and in systems where dashboards and triage must be done without cracking open the production environment for on-box work. Teams need to understand and verify the lineage of inbound requests that end up in data stores and in outbound calls or notifications to other systems. Lineage and Observability NFRs are the requirement that creates the need for inter and Intra team correlation capabilities.  Everyone should have tracing and monitoring Non-Functional Requirements (NFRs) that describe their observability needs. Those NFRs should describe how a system must support tracking work through a system from the time it enters until the time it transitions to at rest or the time it communicates with other systems.  Teams without these NFRs often end up scrambling to provide production metrics and debugging tools during production events. YouTube Vid...

  Quit bundling common or platform work with your user features and use cases. Get better flow management and predictability by breaking out unrelated work. Recognize "I'm in there anyway" as an antipattern. Video on YouTube Speaker's Notes To be added

Python Pandas are targeted at data science applications but they are useful for everyday data conversion. I needed to convert a wide column sheet of NFRs in a TSV to Markdown for display. It was easy with just 4 lines of code This code  Opens the delimited file,  Fills in all the empty cells with empty strings,  Writes out the .md file df = pd . read_csv ( args .csvFile, engine = "python" , sep = args .sep, header = args .header) with open ( args .mdFile, "w" ) as md :     df . fillna ( "" , inplace = True )     df . to_markdown ( buf = md , index = False ) Usage Example https://github.com/freemansoft/Non-Functional-Requirements Complete Program w Command Line Arguments __doc__ = """ This converts a delimited csv file to a markdown table using pandas. Run with the -h option to see arguments """ import pandas as pd import argparse csvFileDefault = "NFRs.tsv" mdFileDefault = "NFRs.md" headerRowDefau...

Cloud Key/Value databases have some interesting features and limitations that can change the way we model our databases. There is a class of key/value stores that have native change feed support that is in a form that is easy to connect to and operate against. In CQRS, we capture an event stream in a primary store and then materialize that in a query store.  An alternative to the CQRS pattern is to create an updated version of a document and then append that updated version of the document to the database.  We're going to look at the drivers and patterns for the latter approach. Video Presentation Content  Speaker's Notes to be added later Published 2022/10