Blog de Joe Freeman

Modern development is surprisingly complicated.  My personal software development environment is suffering a severe case of urban sprawl.  While working on some Container IAC scripting for ML and the cloud, I had to pop into a Linux environment. This environment map made me realize that I have a crazy set of different specialized sandboxes. Software systems continue to grow and become more complicated. Software Engineering platforms growing right alongside them. YouTube https://youtu.be/67i43rBkk1c Why this page exists This page exists as a link root for the video and to make it easy to refer to this diagram. Revision History Created 2024 08

There is a great shift left  movement in software engineering that is in some ways the natural outcome of the Agile movement.  Everyone wants to push more power and responsibility onto engineering teams removing roadblocks and shortening cycle time for making changes and creating products.  This increases the number of tasks and cognitive load on those teams as a tradeoff for shorter cycle times and other concerns. The idea is that you push ownership of every facet of a product to the lowest level possible.  People often use Amazon AWS as an example of success in this area.  Originally shift-left was primarily an engineering function where you pushed all of the technical responsibilities to a single team.  The team creates better products because they own the entire lifecycle and any of the cycle times.  Organizations have eliminated adjunct positions creating a staff single developer type that owns the previous test, DevOps, CI/CD, data, and all devel...

Create resources in Azure using the Azure Portal Template UI. We created a resource group with a Namespace and individual EventHubs in another video. Then I exported the resource group contents to a JSON file. Here we load that JSON file into another Resource Group to recreate the EventHubs and their associated identities and security settings. The code for this  discussion came out of working on a different blog posting .    The exported template is mostly hardcoded. You will want to parameterize any names that might vary by environment by reuse of the template for other purposes.

One of my favorite features of Azure DevOps is its ability to integrate Work Item updates with Office Excel.  This means you can run any canned ADO query from Excel, update the retrieved work items and then publish the updates back to ADO all from inside Excel.   Microsoft Guidance See the following pages https://docs.microsoft.com/en-us/azure/devops/boards/backlogs/office/bulk-add-modify-work-items-excel?view=azure-devops&tabs=agile-process https://docs.microsoft.com/en-us/azure/devops/boards/backlogs/office/faqs?view=azure-devops Install Azure DevOps Automation Download and install Microsoft Office Integration tools by installing 2019 dev-ops automation   This installs a TFSProtocolHelper that ADO can use to open Query results in Excel Create an ADO query that will execute and fill Excel Click on the Queries  in the Boards Menu Create a Query and save it to create a named query I tend to create two-tier queries for Features  first-level children,...

Machine Learning and Machine Models are the latest wave of change in the software development and business rule industry.  We spent the last 20 years continually improving our Software Development Lifecycle resulting in today's Continuous Integration and Continuous Development (CI/CD).  Machine Learning, Build and Train and ML DevOps are just different enough that we need to step back and rethink some of our current standards. Using standard CI/CD hardware and software for ML build and train may not be the right approach. Feature Creation and Model Development  Feature development and Model training are iterative processes with tens or even hundreds of train/analyze cycles. Data scientists need the flexibility to make rapid changes and the compute support to do these iterations in some reasonable amount of time.  Regulators and Model Risk Officers need to to see the data transformations and training data itself  understand why a given model comes up with its res...