Visualizing the Donors Choose data set with Kibana and Elasticsearch

The Elasticsearch example codebase includes a Donors Choose public data set. The example uses a set of Kibana visualizations. The following image shows a subset of the visualizations used in the dashboard.  

Donors Choose Kibana Dashboard

  • The map visualization uses provided geopoint , Lat and Long, data.
  • You can see there are 
    • 6.2 million donations in the data set.
    • 2 million donations.
    • $500 million in donated
    • Contains data from 2003 through 2018

Video Talk

This talk mostly describes how to get the data set and index it in Elasticsearch and then visualize with the provided dashboard.

Importing the Dashboard

This assumes that you have already indexed the data using the scripts in the GitHub repository. See the related blog pieces for more information.

Connect to the Kibana dashboard.  If you ran Elasticsearch / Kibana locally then the URL is probably:
  • http://localhost:5601
Verify the index exists.  You can explore the fields by clicking on donorschoose
Visualizations bind to index patterns.  Create an index pattern using the new index.
First specify the pattern value you will use to bind to the index
Specify the field to be used for the time series. All Elasticsearch data is time series data.

You should be presented with the fields that make up the donorschoose index pattern fields
The index pattern donorschoose should now appear in the Saved Objects list
Now we are going to load the saved dashboard configuration.  Load donorschoose_dashboard.ndjson from the git repository
The Index Pattern Identifier will have changed so we need to tell the loaded items that they are bound to the index pattern we created above.  Make sure you do this for any/all that are pointed out in the right side panel.  Select donorschoose for any index pattern binding requests.
The dashboard consists of a Dashboard parent and 30+ visualizations. Find and click on the Dashboard parent to launch a connection to the dashboard.

Tha's all folks. You should now see a visualization similar to the one at the top of the blog. Set the date range to start in 2003.

Blogs in this set

Comments

Post a Comment

Popular posts from this blog

Accelerate Storage Spaces with SSDs in Windows 10 Storage Pool tiers

Image
Create Storage Spaces in Windows 10 Windows Server O/S contains Storage Spaces support for Server Spaces tiered storage. You can front slower spinning disks with smaller faster SSDs. Windows 10 has a Storage Spaces GUI Control Panel that does not include the tiered storage GUI. This means Powershell must be used for all configuration. https://github.com/freemansoft/win10-storage-spaces  contains scripts that create tiered storage pools that integrate SSDs as caching drives and HDDs as storage drives. They assume you have at least one SSD and one HDD. The scripts automatically find all raw drives and add them to the pool. Some HDDs have their types incorrectly identified. The script can coerce them to be MediaType:HDD The entire virtual drive is added to the system as a single large volume You need at least 1 SSD and 1 HDD to run cached storage / Simple resiliency 2 SSD and 2 HDD to run cached storage / Mirror resiliency / 1 SSD and 2 HDD to run cached storage / Simple re
Read more

Docker on a Chromebook on Crostini - Neverware CloudReady is ready

Image
Chrome OS is not just for kids and coffee tables. It can provide a useful Linux platform too. Crostini brings a secure linux environment to Chrome OS.  Sophisticated development and deployments can be created with Docker containers running inside in this environment.  Chrome OS is capable of running Anaconda, Jupyter notebooks and other containers on many different developer machines. I run CloudReady Chrome OS on a Dell E7x40 to great effect.  References Chromebook firebase setup guide Cloud Ready running Docker without Crostini or VirtualBox Crostini forums on reddit Install docker-compse on Ubuntu Run Linux programs on ChromeBook with Crostini Hackernoon  Pixelbook Revisited: Running Docker Containers Running the shell and connecting Chrome to docker apps Install and run Docker in Crostini All testing was done on CloudReady running on a Dell E7240. I was able to run anaconda/Python and other docker-compose using docker-compose.yml files my github that I had
Read more

Upgrading the HP Chromebook 14 (Falco) SSD

Image
Walmart sells an HP Chromebook 14 with T-Mobile 3G for $350.  The unit comes with 4GB of RAM and a 16GB SSD.  The SSD is way more than you need as an average Chromebook user.  Microcenter currently has the same Chromebook without the T-Mobile 3G for $250 with a 16GB SSD and $270 with a 32GB SSD.  Linux heads and other nerds, like me, always want to upgrade their devices. I recommend that you just a unit with the size SSD you think you can live with.  Upgrading a Chromebook 14 NGFF SSD is not for the faint of heart .  There are a bunch of small cables and at least  13 screws on the back and 3-5 inside that have to be removed. The keyboard is clipped into the base of the unit.  Those clips must all be worked loose.  The motherboard with the SSD on it is held in via 3-4 screws, the heat pipe linkage  to the CPU behind the fan and the USB port openings on the left hand side Of course I didn't listen to my own advice and bought the Chromebook because it is pretty much a laptop
Read more