Hello there!
For those who might not know me, I’m Ryder, the lead programmer for 33691A.
With Worlds being virtual last year, I ended up having a fairly significant amount of time to work on a large variety of robotics projects that didn’t necessarily pertain to the robot itself, but rather made the VEX development process a bit less gruesome. One of these projects was the PROS Grafana Library, a project designed to allow statistics about the robot to be easily recorded by a V5 brain and in real time be sent to a Grafana (visualization) dashboard.
The PROS Grafana Library consists of three parts:
- The PROS C++ template
- The PROS-Grafana CLI, a command-line interface allowing interaction with the brain
- (Work in Progress) The optional Grafana plugin allowing custom visualization of data such as absolute positioning
I’ve gone ahead and divided this post into not only the latest version of the project but also the inspiration and iterations throughout the last six months. You can expand each section by clicking on the arrow next to it.
Motivation
Motivation
One of the most common annoyances teams have when building their autonomous program is tuning PID Controllers. Although some might tune theirs through trial and error, a more effective way is by looking at a graph of the chassis’ velocities and using something like the Ziegler-Nichols method to at least start with a good approximation of the values. Unfortunately, this usually leads to either printing out the values in the terminal or graphing it on the brain, with both having significant disadvantages.
Early Development
Early Development
The beginning of this project went fairly smoothly, although it was extremely convoluted and required an MSI installer, custom PROS CLI fork, C# program, and C++ template to work.
First GUI Iteration:
First GUI Iteration - YouTube
Second GUI Iteration:
https://youtu.be/iYkdGP4fm1g
The issue with this design
To explore the disadvantages of the old GUI, we have to explore how it operated. The system utilized something known as a Named Pipe, a method of transferring data between two programs on the same computer. As the visualizations were done on a C# app, a named pipe was created between the custom CLI fork and the C# program to achieve this result. An impeccable diagram detailing this process can be seen below:
However, the following issues after around two months of development were found:
- Although the C# visualization could handle super fast speeds, it began to struggle as the amount of data being sent through increased
- The PROS CLI would be completely overwhelmed by the amount of data being sent through if it wasn’t a wired connection
- Manually making all of the visualizations was becoming a hassle
- There were far too many parts of the process to maintain
- There was no persistent data collection, meaning everything would be lost as soon as you stopped the program
- The C++ side was far from intuitive
- Only supported on Windows, which is particularly inconvenient when half of your organization lives on Apple products
This led to a replanning of the entire workflow to be more robust and provide an easier way of actually using the program, both for me and the end-user.
Current System
Current System
One of the largest issues of the previous iterations was having to manually program the visualizations. The absolute positioning display alone took over 400 lines of code after a bunch of abstraction, making it less than ideal to continue adding more ways to show data.
This is where Grafana comes in. Grafana is an industry tool for visualizing data that already comes prepared with charts, gauges, bars, and a bunch of other neat gadgets that lessens the amount I have to program substantially. It became clear that Grafana was needed in the workflow, and so the process changed, this time not only ditching the C# pipeline but allowing persistent storage:
Data Storage
An important factor in gathering data about your robot is having as much of it as possible. The default interval for these packets of data to be sent to your computer is 20ms, meaning 50 packets of information are sent within a second. For a traditional .txt or .csv file, querying thousands upon thousands of rows of data will suffer greatly performance-wise, requiring a more efficient solution: SQL.
SQL is a common querying language mostly used in MySQL or PostgreSQL databases. While these are performant and are great for massive companies or server racks, they are inconvenient to setup on a home computer and are a bit overkill for what would be stored in this scenario. Thankfully, SQLite keeps a lot of the performance benefits of SQL while remaining a single .sqlite file, something that requires no infrastructure whatsoever.
As the custom CLI receives data from the brain, it is then piped into this SQLite in the form of a table, where it contains the various attributes being tracked by the program while also recording the time the data was sent. This is crucial, as otherwise Grafana wouldn’t know where to the points on a time series. An example of this format can be seen here:
This data is then read by Grafana automatically and made available to the end-user, where they can then arrange charts in whatever fashion they’d like.
Getting Started
Getting Started
Although the documentation will likely be revised, you can find the instructions on getting started in the following GitHub repositories:
Contacting
Although I’ll try to be as active as I can in responding to any questions or concerns on VexForum, I’m far more active on my Discord: Yerti#4849
Ending Note
I tend to post development updates fairly regularly on the VTOW discord in case anyone wants to see in further detail the process in which I ended up making this tool and many more.
As always, suggestions are absolutely welcome and encouraged. Thank you so much to the PROS developers who have helped guide me along with this project!