Build Test Adapt Check - When will I know if it works

Starting with a good foundation puts us in a position to step into a build/test/check/adapt iteration loop that drives an expanding and improving product or experience. The key is to figure out how to measure success from the customer, organization, regulatory or privacy, and other non-functional requirements perspectives.

We want to optimize the Build - Test - Adapt - Check cycle in order to make good changes with a short cycle time.


Hopefully, I'll add the content of this talk in the future.  Until then you have the diagram and the video.

Video

Here is the video where I go over the diagram.

Revision History

Created 2024 11

Popular posts from this blog

A full context breaks three local models, each in a different way - 7

Image
In freemansoft/Flutter-AdaptiveCards a demonstration Dart chat server hands a question to a local Ollama model. It asks for the answer as Adaptive Card JSON, a strict, closed-vocabulary schema that a Flutter client renders as interactive UI rather than as text. A set of probes in that repository puts identical questions to different local models. Probe context_fill_probe.dart  produced every figure in this article. It runs 25 test cases, one question each. A test case passes only if the reply used an element type that would answer the question. Every other probe in that set had been asking into a nearly empty window. That window holds a system prompt, one question, and at most a short canned exchange showing the model the shape of a card. The literature documents that a long context degrades model behavior. What it does to a model's adherence to an output format is measured less often. The test probe simulates a long conversation by filling three q...
Read more

Installing the RNDIS driver on Windows 11 to use USB Raspberry Pi as network attached

Image
I do a lot of my development and configuration via ssh into my Raspberry Pi Zero over the RNDIS connection. Some models of the Raspberry PIs can be configured with  gadget drivers  that let the Raspberry pi emulate different devices when plugged into computers via USB. My favorite gadget  is the network profile that makes a Raspberry Pi look like an RNDIS-attached network device.  All types of network services travel over an RNDIS device without knowing it is a USB hardware connection. A Raspberry Pi shows up as a Remote NDIS (RNDIS) device when you plug the Pi into a PC or Mac via a USB cable. The gadget  in the Windows Device Manager picture shows this RNDIS Gadget connectivity between a Windows machine and a Raspberry Pi. The Problem Windows 11 and Windows 10 no longer auto-installs the RNDIS driver that makes magic happen. Windows recognizes that the Raspberry Pi is some type of generic USB COM device. Manually running W indows Update  or  Upd...
Read more

We tried 14 levers to get reliable SDUI card JSON from a local LLM model - 2

Image
In freemansoft/Flutter-AdaptiveCards a demonstration Dart chat server hands a question to a local Ollama model, asks for the answer as Adaptive Card JSON, and a Flutter client app renders the reply. Everything below is an attempt to make that card generation more reliable and more faithful to what was asked. The figures are transcribed from ModelBehavior.md , a lab notebook in that repository. Terms used in this article The first two describe the ledger, the rest name what the levers changed and how each was scored. Probe and flag names are the repository's own. Term What it means here Lever One change tried against the card failures, recorded in the notebook's tuning ledger with its outcome and evidence. Fourteen are counted here. Kind The layer of the system a lever touches: context assembly, meaning what precedes the question in the mes...
Read more