Working with state in MicroPython Timer callbacks

MicroPython supports hardware and software timers with callbacks. Timers can be configured as single-shot or periodic events meaning they trigger only once or on a periodic basis. You can bind a Python function to the timer running in either mode.  Your function will receive a callback whenever the timer rolls over or expires. 

Hardware timers are bound to the actual CPU timers and typically correlate 1:1 to hardware devices.  This means the hardware timer configuration is hardware specific to the size and capabilities of the hardware timers.  In general, you can only tie one callback to each timer because the callback is bound to the hardware timer interrupt handler.  

MicroPython also supports software timers. Software timers are the only option on CPUs like the ESP8266 where hardware timers are scarce or are dedicated to other functions. They have the same callback/handler restrictions as hardware timers.

Video Content

Related

  • https://joe.blog.freemansoft.com/2022/12/associating-state-with-micropython.html
  • https://joe.blog.freemansoft.com/2022/12/micropython-is-single-threaded-but-it.html
  • GitHub: https://github.com/freemansoft/ESP8266-MicroPython

Initializing a Timer

Timers are started with an init() call.  You can specify the timer behavior and callback function as part of the init() call.  The callback function() will be invoked at the end of every timer period.

Timer.init(*, ..., callback, ...)

Some timers also have an API that lets you set or change just the callback function with a Timer. callback(callback_func). Timer APIs may vary slightly by IoT device type so check the documentation for your device.

Callback Parameters

MicroPython timer callbacks only include a reference to the timer itself.  No callback-specific data other than the timer reference is made available as part of the invocation. 

def callback_handler(t):

This means that using the exact same bare function as a callback across timers has no way of maintaining different states for each timer. Ex: Each timer is bound to different I/O pins.   

A better alternative is to have separate handler instances for each timer.  An Object-Oriented pattern for this is described below.

Timer rollover callbacks execute as an interrupt handler

No memory should be allocated as part of the callback function invocation. 

Timer callbacks are invoked by interrupt handlers while the primary Python thread is idle. This is probably the reason that the MicroPython timer callback signature is different from the Thread.Timer callback signature in regular Python. 

You are restricted in what you can do in an interrupt handler with respect to memory allocation. This is called out on many of the MicroPython hardware-specific timer documentation pages. Ex: pyb timer callback docs

Note: Memory can't be allocated during a callback (an interrupt) and so exceptions raised within a callback don't give much information

Make sure that you have pre-allocated any memory required for callbacks.  The I/O pin toggling callback example shown below allocates no memory when invoked.

  • Keep the code as short and simple as possible.Avoid memory allocation: no appending to lists or insertion into dictionaries, no floating point.
  • Consider using micropython.schedule to work around the above constraint.
  • Where an ISR returns multiple bytes use a pre-allocated bytearray. If multiple integers are to be shared between an ISR and the main program consider an array (array.array).
  • Where data is shared between the main program and an ISR, consider disabling interrupts prior to accessing the data in the main program and re-enabling them immediately afterwards (see Critical Sections).
  • Allocate an emergency exception buffer (see below).
  • ISR’s cannot create instances of Python objects. 

Example Flow

This describes the setup and execution flow for a Timer driven callback in https://github.com/freemansoft/ESP8266-MicroPython . 
  1. Create a web server that has a reference to the timer and a callback function
  2. The web server responds to requests to start and stop the timed behavior which results in init()/deinit() calls for the timer
  3. When running, the Timer invokes the callback function whenever the timer period is exhausted.  The callback in this case just toggles the state of an LED.
The example can start and stop a timer at will by invoking init()/deinit().  The timer invokes the callback function provided in the init() call whenever a timer period expires. The callback function processes whatever it needs to do.   Periodic timers then reset the period count setting up the next callback.  

Note: We can change the callback function bound to the timer every time we init() it.  


This entire process happens essentially as background work while the web server in the example is idle waiting for requests.

Holding State in Global Variables

Callback functions() only receive the timer itself as a parameter.  They can't allocate memory and they don't get passed in any other state.  It is best to write callback functions so that they don't require any persistent state or state across calls but this is not always possible.  Ex:  you wish to send an MQTT message to the cloud on a periodic basis.  You would need the connection information for the target URL.

Any required persistent state must be allocated outside of the function definition.  The function definition must be able to reference that external memory. We don't know what thread the function will run in and should assume that it may be in a different context than the primary execution loop. 

The example below declares a variable periodic_target I/O pin outside the function so that it maintains its state across callbacks.  The variable is described as global inside the function so that the code knows to use the externally declared value instead of creating one that exists only for the function invocation.


periodic_target = Pin(2, Pin.OUT)

def toggle_pin_callback(t):
    """sutable for call from a timer event"""
    global periodic_target
    periodic_target.value(not periodic_target.value())

Global variables are generally frowned upon and can create unexpected coupling.  They are also a problem if we wish to use the same function for multiple timers. Ex: flashing different LEDs at different rates bound to timers with different periods.

The function described above can be passed as a callback by reference.  The line below shows the function being passed in as a callback handler.

    a_periodic_operator = PeriodicOperator(Timer(-1), 500, toggle_pin_callback)

Holding State in an Object

We need a reliable way of providing persistent storage to the event handler across invocations. Python supports object-oriented programming.   We can define an object that has multiple functions/methods on it.   Then we can create an instance of that object, or struct, with a state local to just that instance.  The TogglePin example below is initialized with the hardware pin we are going to toggle. All references to functions on that object have access to that pin.  This means that toggle_pin_callback() has access to the _periodic_target of its instance. 


class TogglePin:
    """A sample target class that gets invoked as a timer callback"""

    def __init__(self, pin):
        self._periodic_target = pin

    def toggle_pin_callback(self, t):
        """the callback method"""
        self._periodic_target.value(not self._periodic_target.value())

The pin is bound to a specific TogglePin instance.  We can create as many TogglePin instances as we wish all bound to different I/O pins. Each instance has its own private state.

Passing a Function on an Object for Callbacks


    a_periodic_handler = TogglePin(Pin(2, Pin.OUT))
    a_periodic_operator = PeriodicOperator(
        Timer(-1), 500, a_periodic_handler.toggle_pin_callback
    )

Longer Running Operations

Check the docs on how to schedule work rather than doing long running task inside the callback.

Timer Callbacks and Interrupts and the REPL

Interrupt handlers continue to run after you break into the REPL.  You can easily see this if you have a Timer callback running when you open the REPL.  The main program stops running but the callbacks continue to be invoked until the timer is terminated with deinit()

YouTube Video

GitHub Repositories

The code samples all came from the version of this repository as it existed at the time of writing this document https://github.com/freemansoft/ESP8266-MicroPython

Revision History

Created 2022 12

Popular posts from this blog

My blog whispered 'genius'. Turning the content into a video shouted 'chaotic dreck'

Image
 Your content is only as good as it presents 💬 I've had plenty of blog articles that I thought were good enough to put out on the Internet. It was only after I tried to use them as a basis for a video that I realized how chaotic or incomplete they were. AI-Assisted Timeline 2 hours wall clock getting an LLM to create outlines for 6 blog articles 1 hour to generate a blog article from the outline 2 wall clock hours of iteration and ideation and proofreading 4 days of coffee decompression Return to project 6 wall clock hours of iteration with the AI and proofreading Published the article. Immediately found problems 2 wall clock hours of iteration and republished 1 day later LLM found problems while working on other article. 1 wall clock hour of proofreading and iteration 4 days of other stuff including coffee decompression Started video recording Unrecorded hours of swearing 1 wall clock hour of iteration and republished Unrecorded hours of moaning and banging head on table. 2 wall ...
Read more

Installing the RNDIS driver on Windows 11 to use USB Raspberry Pi as network attached

Image
I do a lot of my development and configuration via ssh into my Raspberry Pi Zero over the RNDIS connection. Some models of the Raspberry PIs can be configured with  gadget drivers  that let the Raspberry pi emulate different devices when plugged into computers via USB. My favorite gadget  is the network profile that makes a Raspberry Pi look like an RNDIS-attached network device.  All types of network services travel over an RNDIS device without knowing it is a USB hardware connection. A Raspberry Pi shows up as a Remote NDIS (RNDIS) device when you plug the Pi into a PC or Mac via a USB cable. The gadget  in the Windows Device Manager picture shows this RNDIS Gadget connectivity between a Windows machine and a Raspberry Pi. The Problem Windows 11 and Windows 10 no longer auto-installs the RNDIS driver that makes magic happen. Windows recognizes that the Raspberry Pi is some type of generic USB COM device. Manually running W indows Update  or  Upd...
Read more

On Ollama, memory type and runner decide whether a new conversation reuses the cached prompt - 8

Image
This article was revised with later test data. Earlier figures and tables on this page were replaced. Each Ollama runner keeps a prefix cache that can skip re-evaluating a shared system prompt. Nine of seventeen local builds reused the cached system prompt on nearly every new conversation. Four others re-evaluated a 1,024-token batch of it each time. Four more re-evaluated all of it once per system prompt, up to 37 seconds on the largest. The model's memory type and runner set the group, not the prompt. In freemansoft/Flutter-AdaptiveCards a demonstration Dart chat server asks a local Ollama model for an answer as Adaptive Card JSON. That is a strict, closed-vocabulary schema, which a Flutter app renders as interactive UI. The card system prompt that describes the element vocabulary runs to about 3,800 to 4,400 tokens, by the model's tokenizer. The chat server replays up to ten prior exchanges on every turn. Each Ollama runner keeps a prefix cache, so a request whose open...
Read more