Monitor Internet Broadband service with a Raspberry Pi 4 and some Python

You can easily automate capturing broadband connection statistics with some Python code running on a Raspberry Pi, a Mac or, a PC.  I used a Raspberry Pi 4 as my test appliance because it is cheap and can support 1GB/s ethernet connections. That means it is fast enough to service most residential or low-end commercial connections.

I'm lazy and wanted the data to end up in a secure public cloud that could be populated and viewed from anywhere. We can send our broadband statistics from 1 or more locations and graph the different locations against each other. Any tool could be used.

Monitoring One or Compare Two 

We wanted to compare two different internet provider's service levels.  One provider is a FIOS 1GB down / 1GB up.  The other is a cable service with 1GB down / 50MB up. The providers and the technology were different.  We wanted to know if the complaints about one of the providers were valid.

Relies on Speedtest.net infrastructure

We're going to leverage the popular SpeedTest.net infrastructure and services. They provide measurements for
  1. Ping time
  2. Download speed
  3. Upload speed
It turns out that the amount of time it takes to set up the test also varies by the quality of the internet connection and those of nearby servers.   We will capture
  1. Test server selection time

Two different sites running connection and speed tests. Test results are fed to Azure application Insights

Video Talk

The Code

The code runs anywhere you can run Python 3. It has been tested on Raspberry Pi models 400 and 3.  You can find the configuration scripts and code at

Application Flow

This diagram describes the calling sequence and the roles of each tier.
  1. Get the active server.
  2. Search a list of servers for the fasted/nearest one
  3. Run ping tests
  4. Run download and upload tests if they have been requested


Test results and certain log messages can be sent to Microsoft Azure Application Insights. This gives you access to visuals and graphs without having to install or run a visualization server.

The Dashboard!

Speedtest Ping Times

This Azure Application Insights chart shows two things.
  1. FIOS ping time is about 1/8th of the ping time of the cable company.
  2. FIOS ping time was pretty consistent.
  3. Cable ISP ping times are inconsistent.  They can have significant outliers.

Speedtest Upload Times

We see the expected upload time differences between cable and FIOS.   Cable service is asymmetrical with slower upload speeds.  FIOS bandwidth is the same in both directions.

Speedtest Download Times

Both services claim to provide the same 1Gb/sec download speed.  You see here that the RPi can see almost 1GB for FIOS.  Mediacom cable never reaches 1/3 of their rated download speed in this 4-hour test.  750Mb/s -vs- 275Mb/s

Monitoring Speedtest Setup Performance

I noticed that test startup time can vary widely by the provider and by general internet contention.  The Python code captures that setup time. You can see here that FIOS can find the best test server in about 300Msec.  The cable provider setup time varies wildly between 1000msec and 8000msec in the 12-hour window shown below.

Future Additions

DNS lookup times can have a big performance impact. I'd like to add DNS lookup timing as a metric. I'm not sure how to avoid the local DNS cache.

References

Python Metrics and logs in Azure Applications Insights
  • http://joe.blog.freemansoft.com/2021/03/querying-python-logs-azure-application.html
  • http://joe.blog.freemansoft.com/2021/03/querying-python-metrics-custom-tags-as.html
  • http://joe.blog.freemansoft.com/2021/03/displaying-python-metrics-in-azure.html
Microsoft Guides
  • https://docs.microsoft.com/en-us/azure/azure-monitor/app/opencensus-python
  • https://docs.microsoft.com/en-us/azure/azure-monitor/app/create-new-resource
  • https://github.com/census-instrumentation/opencensus-python/tree/master/contrib/opencensus-ext-azure
Python Data generator 
  • https://github.com/freemansoft/speedtest-app-insights
Created 3/2021

Comments

Popular posts from this blog

Accelerate Storage Spaces with SSDs in Windows 10 Storage Pool tiers

Docker on a Chromebook on Crostini - Neverware CloudReady is ready

Java 8 development on Linux/WSL with Visual Studio Code on Windows 10