Patterns for moving data off of a Microcontroller to somewhere else
Microcontrollers can operate independently in complete isolation. They can operate independently while sharing data with dashboards or other applications. They can operate under guidance or based on information provided to them by external systems. They can coordinate operations with other controllers, or remote applications by passing data back and forth.
Let's break this down and talk about moving information from an IOT to remote systems or other devices. We're talking about information transfers from a microcontroller to other devices, applications, and data sinks. Transfers can be self-initiated or externally initiated. Both approaches have their advantages and disadvantages.
Video
Related Blog Articles
Microcontroller-initiated data transfers (Push)
Microcontrollers can push data to remote systems and data sinks. The MCU generally initiates the connection. The MCU initiates the transfer. There are cases where the remote side initiates a persistent connection that the MCU then pushes data across but that seems rarer to me than when the MCU owns connection setup and data transfer. The microcontroller is running the show so it can send data at regular intervals or based on activity.
MCUs can send data as they gather it or they can capture data for some period of time and push the data in a batch. This is driven by the data freshness requirements.
When pushing over WAN or LAN links, every microcontroller must maintain its own outbound configuration and secrets for the remote data endpoints or storage. Fleet administration may need the ability to update credentials and resource addresses.
Sending data as collected has lower latency but with significant overhead. Pushing data over a remote wireless connection may involve more cycles setting up and tearing down a connection than it spends actually sending the data. There are ways of reducing this load like only sending data when the data changes by some amount.
Batching data improves the signal-to-overhead ratio by sending multiple data points at the same time. This delay for more data adds latency to the data ingestion on the other end. There are use cases where data freshness may make a batch the wrong choice.
This diagram shows three main ideas.
- The leftmost shows the typical debugging organization or a hub / spoke relationship where MCUs act as spokes to local servers or machines. IUSB can support multiple connections over the same physical link. A lot of MCU code uses an RX/TX or one of these links to send its debug and console data. USB supports multiple virtual links so some systems open a data-only serial link that is independent of the debugging link.
- The right side is really the cloud and IOT fleet type of view. Individual MCUs send their data over wan or LAN links to cloud or data center services. These calls can be lightweight or fairly complicated like pushing data over TLS links.
- The link at the bottom of the MCU just calls out that microcontrollers can create mesh controller/controlled relationships between just a set of microcontrollers. They may be point-to-point, in a mesh, or in some other topology.
Externally-initiated transfers (Pull)
Pull operations are initiated remotely from the MCU and transmitted as commands or queries. The external system sends a command or query to the MCU which then gathers the current or buffered information and returns it as a response. All of these will be command/response or request/response. The MCU must be continually listening or listening often enough to not miss messages.
Usually, the remote side initiates the connection. You could instead have a situation where the IOT device opens and manages the connection. The remote side sends commands over the persistent connection as it wants data.
Certain types of external data targets shown above are passive and can't participate in this model. They aren't great transaction initiators. Data lakes and event hubs are two good examples. This makes the pull model cumbersome adding extra components.
Microcontrollers may be installed in locations that won't open firewall holes or loosen other protections This makes them hard to study with a pull model. Ex: Some sites may not let external systems reach into IOT devices for data. Remote sites that reach into the MCU must have valid routes into this network the MCU is on.
- The left section is once again primarily focused on a host/client relationship. These are all basically Command patterns where the PC asks for data by a signal or command.
- The right side focuses on enterprise or cloud-type data transfers without an attached host. The right-hand side is a lot less crowded than in the Push model. Transfer can only be initiated by services or tools that can create outbound connections to the MCU. The protocols can be simple as some type of simple serial command pattern or they could be something as complicated as a TLS GET service.
- Once again we can have a mesh or at least a pairing of microcontrollers with each other where data is transferred between microcontrollers. Sometimes that is the entire system. Other times they may be arranged in some type of star configuration where one node is connected to the cloud or a host. You see this type of network with some of the IOT home networks where the devices communicate with a local hub that connects to the internet.
Security
The MCU must have secure endpoints for wireless connections and ways of changing the authentication credentials stored on the device.
This section is to be expanded at some future date or pulled into its own article.
Video
To be embedded
Revision History
2023 02
Comments
Post a Comment