Introducing Overclocked scouting/stats site

Hello VEX teams,
Since last August, members of team 246A have been busy working on an online dashboard that provides match results and robot statistics from the qualification matches at a given competition. The website is fairly similar to that made by Gael Force (http://scouting.dhsrobotics.com) except that our primary focus has been on creating a metric of the performance of a robot in the qualification matches that is not swayed by the quality of a team’s schedule. Our offensive power ranking (labeled mean estimated contribution) is based on how much your alliance scored minus how much it thinks your ally scored so your ranking is unaffected by your allies. Using the offensive data, our program is able to figure out how much each alliance would be expected to score so it can make scoring predictions/estimates that guess the winner with 75%-85% accuracy, and figure out how much a team can reduce it’s opponent’s score for a defensive rating. One of the biggest issues with the algorithm is that when teams score for their opponent’s to get schedule points, the bad teams look better than they are, and the good teams look really bad at defense. For more information see the information on the webpage.
One of the features I just added is that teams can now submit a description of their robot which will show up when other teams look at the statistics for a team. If you summit a description at http://vex.buarobotstats.com/submit_description.html, your team’s submitted design description will be entered into our data base. This is kind of like a robot social network in that it will be more helpful the more teams submit descriptions, so we encourage you to encourage other people to submit descriptions.
Also, some of you may remember how the robodragons scouting database had scoring breakdowns last year. The scoring breakdowns were in the csv files last year, but have not been posted to robotevents.com since worlds last year. If anyone from VEX knows why this is, or could fix it, I will happily incorporate that data on the site.

In the works:
1: Data on a team from throughout all competitions
2: Elimination/alliance data. I hope to make tables of elimination matches and a dynamic elimination bracket.
3: Ability to submit youtube videos of matches to be viewed embedded in the page for that match for viewing the match along side the statistics.
4: Links to other sites for a competition or team such as robotevents.com and the Gael Force Scouting database
5: Faster loading times
6: Add skills scores
7: Add searching
7: Things you suggest

So far we only have competitions from 2014, but we hope to add the fall events in the coming weeks.
Here is the site: http://vex.buarobotstats.com

1 Like

Can I just say that I find the ability of this thing’s to predict match results eerie? I’m poking through our results for Saturday’s competition, and your system correctly guessed every match. It’s messing up on rankings, but that’s likely just due to a small amount of data. And it correctly labeled our broken robot as being a burden on its alliance. This is really cool.

If the location setting was by state or province, that would be helpful. I have a hard time remembering what city all of our competitions are in, but I’m 100% sure they’re in Arizona.

How often does this pull data from Robot Events? If it’s once an hour, we could use this to predict how our matches at a competition will go. See if we can’t confound expectations.

EDIT: My teammate is currently complaining because he feels his robot wasn’t ranked as high as he feels is fair by your database. Can we get the equations you’re using here? I’ve seen the one for OPR, but the others are new to me.

EDIT 2: Okay, a few things. First of all, your robot submission page has a subject area labeled “Climb.” I have no idea what that could be referencing. In fact, I don’t really understand why most of those fields are there. Maybe you should rethink how those are set up. Design/Autonomous are the only things anyone really needs to know about a robot.

Second, we’re curious how these match expectations are calculated. Are the aggregate through the season, or are they only checking results from the one competition? Both approaches have their advantages, but utilizing results for a single competition makes prediction for the first half of the day entirely impossible.

You know how in Eclipse when you hover over a function the comments immediately before it come up in a popup? Doing something similar for the various abbreviations would be helpful.

Finally, please DON’T allow teams to just start submitting bios without some sort of confirmation process. Maybe one like Reddit has on /r/IAMA. Insist on a photo being taken with someone holding a piece of paper that says “Hi Overclocked” with their team ID plate or something. The last thing you want is a bunch of angry e-mails from teams who don’t want their information online insisting you remove content.

This looks like something that could be incredibly helpful. It saves me from spending the week before Worlds crunching numbers to try and determine how good all 400 teams are, and ranking them accordingly. That was not fun.

EDIT 3: This is what I was afraid of. You have no screening or verification process. I could, at this moment, go and post information for every single team in Arizona. I won’t, because some of them might not want their design posted online, but there is absolutely nothing stopping me at this moment. It doesn’t even have to be accurate, either. This has the potential to be a real problem.

Also, that “Climb” box I mentioned earlier? I assumed it was to describe your ability to hang in a match? The text I entered didn’t display. And after hitting “Submit” I was sent to this page. That’s great, but after 5 seconds it should redirect me to the home page.

This is possibly the greatest (and scariest) thing i’ve ever seen. Your predictions for our matches are so accurate they’re CREEPY. The only places they’re inaccurate is where our driver decided he’d just descore all of the opponents’ points, so they ended up with scores like 9 or 10 instead of the predicted 25.

But other than that, this is great. Do not ever take this website down EVER

This is really cool guys, keep up the good work!

We are working on the ability to sort by qualification region/state.

It should update every few minutes on Saturdays if there is live updating to robot events. Are hope is that teams will have the data for alliance selection.

I may post the algorithm at some point, but it is over 250 lines of code just to calculate the statistics.

Don’t know why I called it climb. I just changed it to hang to be more clear.

At the moment it is just that one competition, so the statistics aren’t extremely meaningful till the end of qualification matches.

Great idea! I will see if you can do tooltips in google charts.

We have thought a lot about this, and decided that there really is no way of verifying whether a entry is submitted by the correct team. It is configured so that if a team posts a bio, they must use the same email as the bio on record. If somebody enters a lot of spam, it will hopefully be noticeable to me if there are many entries with the same email, but we mostly rely of the community to report spam. If this becomes a big issue, we may need to try and implement a more complicated process.
One of the best ways to defend against this is to ask teams at competitions to submit a description of their robot before any spammers do. Once the email of the correct team is on record, I think the system should work fine.

That is great, Team Overclocked! These numbers will be a valuable addition to compliment the subjective observations/opinions that I gather in matches. Looking back at the past 3 tournaments we have gone to, I would have to say that the numbers reflect my scouting reports very accurately with the exception of a few strong scorers that escaped my (distractible) eye. Those teams that slipped by me come to the surface with your information. This could make chosing a second pick a lot easier. I am looking forward to testing this out at the next tournament we go to. Thanks for your hard work to develop this and to put this into place.

Looks like a great database for information. I haven’t looked at the numbers in much detail yet, but did notice two small problems.

One seems to be a simple mathemagic thing, and the other looks like just a typo.

Kevin
Screen Shot 2014-01-20 at 9.00.45 AM.jpg
Screen Shot 2014-01-20 at 9.00.24 AM.png

I don’t think there is a way to account for random decisions or incidental robot problems. But I think that makes it a great tool for scouting, because a picking team can see that the team they’re scouting had bad or good luck and/or partners in certain matches and shows us how those matches should have turned out if everything went normally.

Just a question, does this site show matches and predictions for tournaments that are in progress? For example, if I’m at a tournament and it’s on lunch break, does this site take data from the morning matches and predict the outcomes of the afternoon matches before they occur?

Even if the answer is no, this is a brilliant scouting site! Thanks for making it and sharing it with us! I look forward to seeing how powerful it becomes with the revisions that you’re planning.

EDIT:
What are the units of the numbers/statistics that you show? Is MEC measured as the average number of points a robot contributes to the total score, or is the number just a relative rating that the algorithm spits out?

Thanks, don’t know how I missed those. :slight_smile:

Just a question, does this site show matches and predictions for tournaments that are in progress? For example, if I’m at a tournament and it’s on lunch break, does this site take data from the morning matches and predict the outcomes of the afternoon matches before they occur?

It is meant to be about the number of points scored, but it can be thrown off by teams that score for schedule points and should be taken more seriously in a relative sense.

Thanks for the quick answer!

I look forward to trying this system out in competition.

Love the website guys!! This will definitely be useful. Keep up the great work!:smiley:

If I enter info on my robot now, can I go back later and edit when I have pictures or videos?

Also, we are in the process of making a new robot completely different from the robot in our previous competitions. Should I enter info on the new or old robot?

This looks like a really solid scouting website. Its pretty obvious your team put a lot of thought into this and I am really looking forward to using it at in the future.

However, it appears that you may have a data retrieval issue. For the date 1/18 there are three events that have results posted on robot events that your site shows no data for:

1st Annual Westview MS Wildcat Robotics VEX Event (missing half of qualifying matches and elimination rounds)
Northeast Wisconsin VEX Robotics Competition (missing some of elimination rounds)
Autodesk Youthville Detroit VEX Qualifier 2 (missing some of elimination rounds and has five finals matches)

On robot events these events say that they are still in progress and they all have some matches missing from the results (and some extra matches your system may not account for). My only guess is that the missing information and possibly the status of the events is messing with your data collection system.

You may also want to add a short description of the statistics in the interface in addition to the in-depth description on your ‘about’ page, but other that that what your team is doing is nothing short of incredible. Can’t wait to see where you’ll take this next.

The Youthville Detroit tournament has 5 finals matches because 5 finals matches were played. The intense round consisted 2 ties, 1 red win, and 2 blue wins.

You can always change the description so long as you use the same email address. Regarding what robot to describe, try not to post a design you are not certain you will have for your next competition. I don’t want potential allies to make decisions thinking you are better than you are if that makes sense.

The system only covers qualification rounds right now, but elimination matches will come soon.

Right, but those three tournaments have results on robotevents just like the ones your system recorded. The only apparent difference is that two of the three are missing some elimination matches and the third is missing all of the matches after 12:00 or so. All three tournaments are flagged as still being in progress on robotevents due to the fact that they are missing match results. Now your system is showing results for the Autodesk Youthville Detroit VEX Qualifier 2, so it would appear that the flag has nothing to do with your system recording results or not, which makes sense based on your explanation of the system.

Either way there’s still at least two tournaments that have results on robotevents that don’t show up on your system. That was all that I was trying to point out.

Ahh, I honestly can’t tell what was happening at the Wildcat tournament from robot events; were there redos or something at the end? Anyways, it seems as though everything is working properly for the 27 qualification matches, and I’m not sure what I can do after that until I get the elimination system working. The Northeast Wisconsin tournament had a simple typo that was throwing off the system (now corrected), and I think that the Detroit tournament is fine now as well. Thanks for pointing those out, and let me know if those or any other competitions don’t seem right.

Everything is set up for live updating tomorrow for any of you who are at competitions that post live to robotevents with a 1 minute max delay. If you have a competition tomorrow, let me know how it works for you or if you have any suggestions! Also, the page is now set up for all competitions in 2014, and ordered like robotevents, so any suggestions there are welcome too.