Easily Implemented Semi-Dynamic Qualifying

I made this suggestion at the end of another thread a few months ago and never got any feedback on it, so I am starting a new thread about this specific idea.

What if the very last round of qualifying paired teams of similar ranking? Just the last round. So, in the last round only, 1&3 vs 2&4, 5&7 vs 6&8, 9&11 vs 10&12, etc.

The main benefit is creating a little more separation between teams with similar records without any complex system that would slow events down. Also, this still allows for high quality teams to be paired with inexperienced teams during all previous rounds of qualifying. It would provide an excellent format to get your last minute scouting done, and who doesn’t want to see the top robots battle it out one last time before alliance selection?!?!

Any thoughts?

I would be an interesting way of pushing SP’s even farther down the list of tie breakers. Make WPs the first deciding factor, APs the second, and then the matched round as the third.

Having a 1v1 tacked on after qualifying matches in a 48 team tournament would add 72 minutes to the schedule, assuming a 3 minute match cycle. This would likely reduce qualifying rounds as facilities are usually time constrained.

Not a 1v1 round. Just the last round of 2v2 being paired based on rankings. So the first match of the last round would be 1&3 vs 2&4. It would add no extra rounds of qualifying and no extra time.

Oops misread. Still adds 36 minutes which would have to be taken away from qualifying matches.

I don’t really see a huge benefit. But it would be a season or two away before implementation…

The ranked match is the last qualifier, it’s not played in addition to the normal 6 (or what have you).

One consequence of this is that you will always have 1 or fewer teams with a perfect record, unless there were 3+ teams with a perfect record prior to this match.

I think that’s a good thing. The tie breakers, as important as they are, are never going to be as accurate as a head to head test to determine the best robots.

@Wombat_of_Doom, thanks for starting this post! We really need your of out of the box thinking to solve the issues we face with the qualification system.

I think it sounds like an interesting idea.
But then your kind of stacking odds against certain teams that aren’t quite in the top but are right under it. I think? Am i reading this right?

Implementing this would be interesting, because it brings up the point that @puzzler7 brought up which is that there will be less teams with perfect records, which is a good change IMO. It singles out the dominant team before the alliance selection, which makes it easier to figure out everyones’ standings.

I didn’t catch that part about it…
So in the last round if you were part of that 1 and 3 matchup, would that be a 1v1 match would you get randomly paired against another pair such as 2 and 4?

Neither. 1 and 3 would always be paired against 2 and 4 (or 1+4v2+3, or something). The same for 5-8, and 9-12, and so on.

If you have exactly 2 teams with a perfect record, then they are 1 and 2. Thus, coming out of the 1v2 (with partners) match, only 1 team will have a perfect record. If you have exactly 3 teams with a perfect record, then 1 or 2 of them will be defeated, leaving 1 or 2 with a perfect record. However, I don’t recall ever going to a regional tournament with 3 or more perfect records prior to the final match.

I’ve seen as many as 5 before, but it was a very large tournament (with not enough qualifying rounds). Still, 2 or 3 undefeated teams is better than 5.

Great idea. The current system does need to be replaced soon in my opinion.

As an event partner, you are all overlooking the necessity to rerun the schedule and print and distribute. Its not just about the number of matches, you are adding more than that. Another thing I would like to point out is a robot with a perfect win-loss record (due to having the gravy schedule during the day) may have a very high seeding (as in 1 or 2) and be not even close to the best robot (or best five robots) at the event. The autonomous points being added as a ranking helped this quite a bit in one way but did take away the in match strategy that the seeding points provided. Good suggestion. Not sure how practical it is.

To be honest I think this proposal is them trying to meet event partners half way. What they really want is to schedule every round of qualifications 1 at a time. Obviously releasing new schedules 7 times is too much work for EP. It would be too chaotic for teams not knowing when to expect matches until right before.

So the compromise is what if only the last round of qualifications is dynamic. Obviously this would create at least some additional burden compared to an entirely static schedule but hopefully little enough it is possible.

Why do you have to rerun, print, and distribute? Just put rank numbers instead of team numbers on the last round of the original schedule and don’t update rankings during the last round. You might need a short (10 min?) break so everyone can find their rank and the final qualifying match that has that rank in it.

I just finished doing a simulation of the current static match schedule and a new dynamic match schedule, and the results were not as big of an improvement as i had expected. It did have the advantage of decreasing the number of undefeated teams and the number of teams who never won a match, but the over all error between the expected results and the final results was negligible.

Pros:

-Less 0 wins - 6 losses
-Less 6 wins - 0 losses
-More 3 wins - 3 losses
-More close matches

Cons:

-Schedules reprinted 6 times throughout the day
-Teams at the start of the list will have less time to prepare for their next match than others
-Negligible improvement in final ranking

Further research:

It may be possible that with a dynamic match schedule, it would take less qualifying rounds to reach ideal ranking than it would with a static schedule. This will need further automation and simulation to determine.
Static vs Dynamic.png

Interesting stuff.

On the chart, what does each plotted point represent? Is it the rank of a single team in a single simulated qualification tournament? (I assume that’s what it is.)

I keep counting the plotted points and getting a different number of blues vs reds. These should be the same, yes? (I only counted them to test some of my assumptions about your plot. I assume I’m miscounting them. I could print it out and number them, but it’s probably more sane just to ask you.)

On the methodology of the simulation, how was the ground truth ranking set? Were there, for instance, any ties in your rank. Based on my guess about what the points represent, it looks like the answer is “no,” but I’d like to know that.

If you didn’t use a dataset with tied ranks, it would be interesting to see that.

Also, in the ground truth ranking, is it simply a strict ordering of teams, 1 through N? Alternately, is there a “goodness” factor, say, from 0 to 1000 with teams distributed unequally along that line?

How did you determine tournament wins? With a “goodness” factor system, you can just compare alliance goodness sums. With strict order ranking you can do that as well, but it doesn’t account for the real-world situation where there are great gaps in a team’s abilities.

Does your round simulation allow ties?

What method did you use to generate alliances?

What is the average ranking error for each of the two systems?

That is correct, each point is a single team’s final ranking.

There should be 64 red and 64 blue, however i did nothing to ensure that a red and a blue in the same position both show up, so it could be that is the problem.

For the ground truth, i generated a list of teams with a normal distribution of strengths, with 50% win rate as the mean, and i believe 20% as the standard deviation. I ended up with one team outside the 0-100 range, at 103 or something and i bumped it down to 95 (due to the next step). I then generated a standard deviation for each team based directly on their base win percentage (scaled such that 0 or 100 are both 0 stdev and 50 had 20, like the generated group. So, in the original data set, there were no ties, to the best of my knowledge.

I did this simulation by hand, which was a bit of a pain, and i am going to automate it to simulate more situations, this was just a first step. I will take ties into consideration, but i dunno how much it would actually effect things. I also did not simulate tie breakers.

I did some plots personally with the uneven distribution, but the plot i displayed here was purely 1-N.

I have been researching some statistically valid ways of comparing and combining teams, and found a relatively reliable method (my last check showed about 65-70% prediction accuracy) To combine the “goodness” of the two Red Teams, it is a sum of the goodness (mean historical win percentage) and root sum square of their Standard deviations. Then to compare red vs blue, it uses the statistical likelihood of red beating blue, based on their combined means and standard deviations. If the probability is greater than 50%, red wins, if it is less, Blue wins.

Ties were not accounted for and would probably have been assigned to blue victories, i would have to check my method of execution. When i automate the process, ties will be handled properly.

I used a random index to shuffle teams for each round (6 rounds) of the static schedule, and a random index to shuffle teams within their most recent records for the dynamic schedule. Specifically, they were ordered first by their N-2 and N-3 round record, second by their N-1 record, and third by the random index.

Due to some less than perfect methods used, the average came out to 0 for both, which just showed that there are always going to be the same number of winners and losers. I would like to look into whether each method favors raising poor teams or dragging down strong teams, but i have not gotten there yet.

Fantastic work. Thank you for the answers. I’ve gone through it once quickly, and I’m going to go back and read it more carefully later.