Joe Freeman: Tech, Teams, Trade-offs

This article was revised with later test data. Earlier figures and tables on this page were replaced. Each Ollama runner keeps a prefix cache that can skip re-evaluating a shared system prompt. Nine of seventeen local builds reused the cached system prompt on nearly every new conversation. Four others re-evaluated a 1,024-token batch of it each time. Four more re-evaluated all of it once per system prompt, up to 37 seconds on the largest. The model's memory type and runner set the group, not the prompt. In freemansoft/Flutter-AdaptiveCards a demonstration Dart chat server asks a local Ollama model for an answer as Adaptive Card JSON. That is a strict, closed-vocabulary schema, which a Flutter app renders as interactive UI. The card system prompt that describes the element vocabulary runs to about 3,800 to 4,400 tokens, by the model's tokenizer. The chat server replays up to ten prior exchanges on every turn. Each Ollama runner keeps a prefix cache, so a request whose open...

This article was revised with later test data. Earlier figures and tables on this page were replaced. A long conversation in the context window lowers structured-output scores on five of eight local models. Two Nemotron 30B models lose 8 of 25 cases each, and a parse check catches only one of their two failure types. Rank models at the context length you will run at, and check each reply against the specific question asked. In freemansoft/Flutter-AdaptiveCards a demonstration Dart chat server hands a question to a local Ollama model. It asks for the answer as Adaptive Card JSON, a strict, closed-vocabulary schema that a Flutter client renders as interactive UI rather than as text. A set of probes in that repository puts identical questions to different local models. context_fill_probe.dart produced every figure in this article. It runs 25 test cases, one question each. A test case passes only if the reply used an element type that would answer t...

This article was completely regenerated using updated prose and later test data. Earlier figures and tables on this page were replaced. See the revision history at the bottom In freemansoft/Flutter-AdaptiveCards a demonstration Flutter client sends questions to a Dart chat server. The server passes each one to a local Ollama model and asks for the answer as Adaptive Card JSON. The client renders the card that comes back. To measure which models manage that, a directory of probes sends a fixed set of questions straight to Ollama, one model at a time. Each probe builds its requests the way the chat server does and judges every reply with the server's own card detector. Some recorded results came from the test setup rather than from a model. A busy machine caused one incident, the Ollama runtime others, the harness itself the rest. Each looked like model behavior. Each rule below stops one such mistake. Five rules come from a measurement that went wrong or near...

This article was completely regenerated on October 4, 2026 from matched Ollama 0.35.1 sweeps of both hosts. Earlier figures on this page were replaced. In freemansoft/Flutter-AdaptiveCards a demonstration Dart chat server hands a question to a local Ollama model. It asks for the answer as Adaptive Card JSON. Adaptive Cards is a strict, closed-vocabulary schema. A Flutter client renders the JSON as interactive UI rather than as text. The project's test probes measure how well fifteen local models produce that JSON, and how fast. The project built the probes on a 64 GB M1 Max, then ran them on a 16 GB M5. Eight of the fifteen models fit, or nearly fit, in 16 GB. On the M5 each model took 1.14x to 1.78x the M1 Max's median time per call. The median of the eight ratios is 1.40x. Each model ran once per host, so each ratio gives a direction, not a per-model figure. The top three models score within one shape-probe case of each other. Memory fit a...