Patterns for Moving data onto a Microcontroller
Microcontrollers can operate independently in complete isolation. They can operate independently while sharing data with dashboards or other applications. They can operate under guidance or based on information provided to them by external systems. They can coordinate operations with other controllers, or remote applications by passing data back and forth.
Let's break this down and talk about moving information onto an IOT from remote systems or other devices. We're talking about information transfers from a microcontroller to other devices, applications, and data sinks. Transfers can be self-initiated or externally initiated. Both approaches have their advantages and disadvantages.
Video
Related Blog Articles
Remote systems can push data to the Microcontroller over wired or wireless links. Microcontrollers can listen on wired and wireless. serial ports for command protocols.
This means that the remote systems must be active having the ability to push data without waiting for a query from the microcontroller unit (MCU). The connection setup and management are different but the central fact is that the microcontroller (MCU) has to be ready to accept the connection stays the same. They can stand up internal web servers to receive HTTP/TLS requests. They support a variety of transports like USB and various protocols on USB including HID, USB MIDI, and others.
In general, it is best to use some standard rather than creating your own protocol. This reduces your work and lets you focus on the use case-specific behavior of your MCU program. Data can be streamed into the MCU. It can be a stream of characters like serial commands or data protocols. An alternative is to use it in a more block-update fashion with something like an HTTP POST.
MCUs receiving data pushes must be ready to accept the traffic at any time. A lot of MCUs are configured to receive on multiple channels at the same time. The typical MicroPython MCU is listening for inbound REPL commands on the USB connection and possibly listening on a network socket for web requests.
We see a few styles in the picture above.
- The left-hand side of the diagram shows directly connected or virtually connected serial links. You will probably need to implement some type of framing on the serial links so that remote systems can determine where a packet start is. USB transport is more complicated with its parallel serial streams. Most folks will implement some standard protocol on top of the USB like CDC serial or HID MIDI.
- The right-hand side describes some of the standard protocols that can be used to capture data sent by remote systems. They could be standard web or other API protocols.
Network receiver and web service examples often have some type of polling loop filling and sending from buffers. You see this construct in a fair amount of code where some listen() method has to be called on a regular basis.
It is possible to accept inbound data on an interrupt-driven basis. Some of the serial protocols do this. Interrupt handlers must be short and succinct and can't allocate memory in some languages like Python. This can drive a model where there is a handler to capture the inbound data request in a fixed-size buffer. Then a loop() style process looks at the buffer and picks up the data from the buffer. You have to run that code often enough that you don't have buffer overflow.
When receiving unsolicited push data, the MCU must be responsive to avoid input overrun or data loss. They must continually listen for inbound requests or catch data with interrupt handlers. MCU programs must empty the input buffers in a timely fashion.
MCU Requests Data
The MCU request pattern is different. The MCU retrieves data when it is ready. The operations are synchronized in the MCU with other operations. The MCU pings or notifies a remote system that the MCU wants data and the remote side responds.
Most people are familiar with the scenario where the MCU wants to pull data down from a website. The MCU opens a connection and makes a request and waits to receive the response. There is zero expectation that the website will just call back randomly. This means the MCU only needs to be set up for inbound data in the Response phase of a request and not all the time.
The request-response pattern feels like a type of batch transfer. The remote system either generates the data on request or captures and stores the data to have it available on request. Ex: Polling a message topic at regular intervals to see if any commands or other info has been published.
It is less efficient and a bit cumbersome for low-bite or low-field messages or simulated streaming.
Certain types of external data sources are passive and can't initiate transactions for the Push model discussed above. In that case, we need to use a pull model. Some messaging systems and cloud storage are examples.
Microcontrollers may be installed in locations that won't open inbound firewall holes or loosen other protections This makes the pull model the only option. Ex: Corporate sites may not let external systems reach into IOT devices for data.
- The left section is once again primarily focused on a host/client relationship. These are all basically request/response patterns where the MCU asks for data by a signal or command.
- The right side focuses on enterprise or cloud-type data transfers without an attached host. Transfer can only be initiated to services or tools that can accept inbound connections from the MCU. The protocols can be simple as some type of simple serial command pattern or they could be something as complicated as a TLS GET service.
- Once again we can have a mesh or at least a pairing of microcontrollers with each other where data is transferred between microcontrollers. Sometimes that is the entire system. Other times they may be arranged in some type of star configuration where one node is connected to the cloud or a host. You see this type of network with some of the IOT home networks where the devices communicate with a local hub that connects to the internet.
Video
To be recorded
Revision History
Created 2023 02
Comments
Post a Comment