Posts

Ollama silently drops a history message larger than its context window

Image
In freemansoft/Flutter-AdaptiveCards a demonstration Dart chat server hands a question to a local Ollama model. It asks for the answer as Adaptive Card JSON, a strict, closed-vocabulary schema that a Flutter client renders as interactive UI rather than as text. A directory of probes measures which local models manage that and how well. Every one of those probes had been asking its question into a nearly empty context window. Each call carried a system prompt, one question, and at most a short seed exchange. A real conversation fills the window. What changes when it is full? Two findings came out of filling it, measured on an Apple M1 Max with 64 GB and an Apple M5 with 16 GB. Both hosts ran Ollama 0.33.3, and the M1 Max results reproduced under Ollama 0.34.0. What the runner allocates follows one rule , with no counterexample in fifty-one runs across the two Ollama versions. It is often not what the request asked for. A history m...
Post a Comment
Read more

Eight measurement rules from a local-model benchmark on Ollama

Image
In freemansoft/Flutter-AdaptiveCards  a demonstration Flutter client sends questions to a Dart chat server. The server passes each one to a local Ollama model and asks for the answer as Adaptive Card JSON. The client renders the card that comes back. To measure which models manage that, a directory of probes sends a fixed set of questions straight to Ollama, one model at a time. Each probe builds its requests the way the chat server does and judges every reply with the server's own card detector. Several of those results looked like something a model did when the cause was the test setup: the machine, the Ollama runtime, the harness or the probe. Each rule below checks for one of those mistakes before it reaches a published number. Five of the eight rules come from a measurement that went wrong. The other three guard against a known weakness in the setup: the per-call time limit, the point in a long run where a model is measured, and what the card detector cann...
Post a Comment
Read more

Ollama's tool channel beats prose for card JSON on every model that calls it

Image
In freemansoft/Flutter-AdaptiveCards a demonstration Dart chat server hands a question to a local Ollama model. It asks for the answer as Adaptive Card JSON, a tree of typed UI components called elements ( TextBlock , Table , Input.ChoiceSet ). A Flutter client renders that card as interactive UI rather than as text. By default, the card comes back in the model's message body, as JSON text inside message.content , which the chat server parses to recover the card. Ollama also offers a second route, the tool channel. The request body declares a render_adaptive_card function and a schema for its arguments, in the standard tool-calling format. Nothing ever runs that function. It exists only to give the model a schema to answer into. When the model uses it, the card arrives in message.tool_calls[0].function.arguments . Ollama has already decoded it into a JSON object, so there is no text for the chat server to parse. We measure the tool channel t...
Post a Comment
Read more

Benchmarking local model LLM generated Adaptive Card JSON on a 64 GB M1 Max and 16 GB M5

Image
In freemansoft/Flutter-AdaptiveCards  a demonstration Dart chat server hands a question to a local Ollama model and asks for the answer as Adaptive Card JSON, a strict, closed-vocabulary schema, which a Flutter client renders as interactive UI rather than as text. We built a set of test probes that measure how well fifteen local models manage that well and how fast. We built the probes on a 64 GB machine and then compared the results to a machine with a quarter the memory. Two machines, one set of test probes Two Apple machines ran the same probes and the same prompts on a 64 GB M1 Max MacBook Pro 14-inch ( MacBookPro18,4 ) and a fanless 16 GB M5 MacBook Air ( Mac17,3 ) used for comparison. The goal was to validate execution and measure performance differences when running the same models on two Apple Silicon chips with different memory sizes and bandwidth. Both hosts ran models back-to-back for hours, so each median below carries whatever position in that r...
Post a Comment
Read more