In The Zone Matches: By The Numbers

Hey! So it’s been a long time since I’ve posted here (college, now work, etc.). I’ve decided to start volunteering at events, though, so I’ll probably be a bit more active again.

Today, I was thinking about the match scores over the season, and using data from VEXdb (and later a private dataset from Jordan), I examined the number of matches a team competes in over the season, as well as the average match scores over time.

The most obvious question to answer is: how many teams have competed this year? The datasets I have reveal that 7508 unique teams have participated in at least one competition match at a regular tournament (not counting leagues, scrimmages, or skills-only events). It turns out that 8 matches is the most common number of matches any given team has participated in, but this sharply drops off on either side:
freqOfMatchCounts.png

(This might remind you of Zipf’s Law or a power distribution — for many datasets, the frequency of an item is inversely proportional to it’s “rank” if you were to sort by the number of times it occurs. Interestingly enough, this appears to be followed by this data set!)

For the fun of knowing, the top ten teams in terms of “number of official tournament matches competed in this season” are:

86868R with 139 matches
8675A with 122 matches
3671B with 119 matches
6741A with 118 matches
6842Z with 117 matches
1045A with 111 matches
1970K with 108 matches
10300X with 108 matches
1375A with 107 matches
1375C with 105 matches

These high match numbers, though, are very rare — to see just how uncommon this is, I produced the following two plots (the second is just a zoomed-in version of the first). They show the percentage of teams with less than (or equal to) x number of matches — by the time a team has completed 40 matches in a season, they’re already in the top 10% of teams!
teamsWithMatchCount.png
teamsWithMatchCountZoom.png

Finally, I plotted the average match score over time. The two lines show the average winning and losing score (red/blue, respectively) over all tournaments on a particular date, and then I smoothed the results with a rolling average over three adjacent tournament days. The shaded region is a band 2 standard deviations wide (+/- sigma), so it’s pretty clear that there’s a ton of noise in these scores.
matchScores.png

I showed this around on the VEX Discord, and it was mentioned that a lot of teams start competing in January, which might explain the sudden drop in average score. So, I decided to only consider matches consisting of entirely “hardcore” teams — which I’ve defined to be teams that have participated in more than 40 matches. (Again, from above, this is the top 10% of teams.)
matchScoresHardcore.png

Only considering “hardcore” teams, we see that the average match score continues to go up throughout late January. It is curious that the margin between the average winning/losing score is nearly a constant 35 points over time. One possibility is that this is due (in part) to the autonomous bonus.

Data I’d love to have: I’d love to see more detailed data breakdowns like there were in prior years (e.g. Gateway/RoundUp) with per-goal scores. This would allow for some “deep dive” analysis into specific strategies, and might even open the door towards prediction of match outcomes based on prior performance.

Anyway — this was kind-of a brain dump of some stuff I was thinking about all day. Any thoughts on the data I’m considering? Do you have requests for other plots or me to investigate some other metric?

Didn’t know I have had the second most matches this year. Kind of cool that we have have also won all except 12 of them.

Yeah, I wish RobotEvents reported individual scored items in addition to total scores, it would make analysis much easier

Time for some more fun plots! Today, I looked at OPR. If this is your first introduction to OPR, it’s a measure of the estimated contribution a team makes to its alliance score. This is useful for determining if a team was “riding the coattails” of a better team or if they were carrying the entire alliance weight. Most often it’s applied in scouting at very large events, or if you want to predict match outcomes. The best OPR resource I found was this CD whitepaper: paper: New Scouting Database from Team 2834 - CD-Media: Papers - Chief Delphi (specifically the most recent slide set).

I computed the OPR for every team at every tournament over the entire In The Zone season. Then, I plotted this for the top 5 teams by number of matches played. Below is the resulting chart:
opr_over_time.png

I’m not sure how useful this is, but at least it’s somewhat interesting to see how teams gain/lose OPR over the season. I was curious if “more matches played” would imply a higher average OPR. Plotting every team’s average OPR this season suggests this is the case, but there’s a lot of noise in the lower OPR range:
OPR_vs_Matches.png

And, for whatever entertainment it’s worth, the top 10 teams in terms of average OPR are:

202Z      81.919686
7505B     70.028775
8675A     69.802652
1437Z     69.689467
7090B     68.825390
7884E     68.375270
7995A     67.686159
6135W     67.543907
86868R    67.222964
7258B     66.925269

(Yes, that’s way too many significant figures, but I don’t want to format my floats :stuck_out_tongue: )

If people would like to see specific OPR-over-time plots or have ideas for more things to do with match data, please let me know! :slight_smile:

–Andrew

Great work.

Here’s a couple of things I can think of that I’d like to see:

How determinative is winning autonomous? (By-match data isn’t available, I believe, but overall AP for a tournament is.)
For the hardcore teams (per your definition) how likely is it that those teams played each other? (So are they playing primarily teams of similar experience, or primarily teams with much less experience?)
How closely correlated is OPR to skills scores?

Consider the ELO rank. (Available here: http://vex-elo-rankings.herokuapp.com, discussed in this thread: https://vexforum.com/t/vex-elo-rankings-2017/43925/1 with some interesting match prediction details here: https://github.com/MadDoctor5813/VexRankings/blob/master/VexRankings/app/rankings/ranker.py )
As with any ranking system, it has some deficiencies. Specifically, it doesn’t do a good job of ranking across clusters. That is, if teams only compete against teams from their own local area, a ranking based on that can’t tell us much about how all teams rank against each other. Can you use match clustering data to compare regional/geographic clusters based on whether teams from one region played teams from another, and use that to improve the ranking system?

For instance, here in Kentucky we know a lot about how our teams rank against each other. We’ve had some amount of play involving out of state teams. Teams come up from Tennessee or down from Ohio and Indiana. We also sent a couple of teams to the Kalahari Classic in Ohio. Is it possible to use those instances of cross-region play to create a better (as in more correlative/predictive) measure than raw ELO?

Interesting to not include leagues. Leagues, IMO, are just multi-day tournaments.

Thanks for the interesting information. Keep up the good work

Thanks for the encouraging words, guys!

I’ve double-checked with @LegoMindstormsmaniac (I’m using the data he scraped), and he has confirmed for me that there’s no auton data this year. :frowning: (So, if anyone from RECF is listening, we’d looooove this data!)

Ooh this is a good question. (And apologies about using “hardcore” — in hindsight, that wasn’t the best word to use, but it was the first thing that came to mind when making the charts/naming variables, haha.) I’m working on creating a graph (as in network) view of how teams played each other in matches, but I’m running into difficulties with so many nodes (even sfdp is starting to choke).

For just a basic numeric result, though, there are 24020 matches in which at least one “hardcore” (by which I really mean “very active”) team played. Of these, only 1980 (about 8% of the 24k) consisted of all hardcore teams. This makes sense, because really active teams are rare, and every such team will likely encounter not-so-active teams at tournaments. I feel like this is somewhat related to the friendship paradox (“your friends have more friends than you” becomes “your match-mates have played more matches than you”).

Correlation matrices are fun! (Pearson correlation is computed below.) The Sum column/row will obviously be highly correlated with Driver and Programming because it’s a linear function of the two, but I left those pairwise comparisons in because it keeps the matrix square.



	 	Driver 	  	Programming 	OPR 	 	Sum
Driver 	 	1.000000 	0.622281 	0.708508 	0.964826
Programming 	0.622281 	1.000000 	0.543653 	0.806180
OPR 	 	0.708508 	0.543653 	1.000000 	0.718098
Sum 	 	0.964826 	0.806180 	0.718098 	1.000000

(Note that correlation ranges between -1 and 1.)

But, also because charts are fun:
DriverVsOPR.png
ProgVsOPR.png
SkillsVsOPR.png

From what I can tell, correlations of ~0.7 are somewhat strong, but not terrific. I’d be happy saying, though, that OPR and the summed skills ranking is correlated. The charts show this intuitively, too.

Ah yes. The Elo system. :stuck_out_tongue: This was actually what I was going to start with when I first was trying to get the data. I’ll look into doing that this weekend when I have a bit more time. Actually, I was watching this video about Elo being applied to a multiplayer videogame — apparently there are alterations on the original chess-based concept that make it more applicable to team-based settings.

For what it’s worth, @LegoMindstormsmaniac implemented an Elo ranking system that achieved around 63% accuracy at predicting match outcomes, while his OPR/CCWM predictor is in the high 70s. I’m hoping the videogame-focused alterations might improve it somewhat.

I’m hesitant to even say this because I might not get around to it at all, but I’ve also considered using some of the basic machine learning techniques I know to try to incorporate raw score, OPR, CCWM, current skills ranks, and past tournament results into some kind of ranking system. Not entirely sure how that would work out, though — state-of-the-art methods would require more data than we have right now I think, but it’s worth a shot. This might be able to take into account cross-region modelling, too.

Yeah, they are, but that’s why I didn’t include them. :stuck_out_tongue: I was wanting to see change in scores/OPR over time, and taking into account the “multi-day” aspect of leagues was just too complex. Also, there were only 34 leagues this entire season while there have been 540 tournaments so far. While that is somewhat significant (~6%), the added complexity just didn’t make it feasible given the time I had… sorry. :frowning: