Thoughts on the new elimination rounds system?

VEX V5 Robotics Competition VRC
161

Again, no, it doesn’t. When you want to examine the affect of something, you want to know how thing behave with and without it without changing other variables. Worlds gives them a situation where they have tight control of the referees and the event to make sure things are done as desired, and as a result is also where they have the best data from past years. So they know how the previous system behaves very well with world-level competitors. Now they want to see how the new system behaves by comparing to the old system. This is best done varying other factors as little as possible. Running at worlds allows for that because of their tighter control and because of better past data. This isn’t about sampling a group of competitors to extrapolate to other competitors; it’s not about comparing competitors at all. It’s about comparing two systems. With approximate the same two groups (world-level competitors), does this system seem to run better than the prior system and, if so, are there any changes that need to be made before rolling it out?

The whole issue of being before the regular season next year is so they’ve had a chance to compare it to the old system before having everyone around the world shift over. That gives them an opportunity to adjust little problems, or to drop it at least temporarily if they need longer to address bigger issues.

162

This is my take (and I could be totally wrong, but here goes). I believe that they did not make this decision capriciously. There has been talk about going to 2 team alliances for a while. I also think that the issue of tanking matches seems to have come to a head this season. And although two team alliances don’t totally eliminate it, it will curtail it quite a bit. That being said, the biggest issue seems to be with the single elimination in the elimination round.
I fully understand the objections that have been made.
*robot reliability issues
*VEX Net connection issues
*field reliability issues
*just pure dumb luck

  • TV broadcast (follow the money) - first time I have heard that
    And from those issues, I have my own reservations about the single eliminations, however this is what I believe
    At the time that RECF put out the Agenda for this year’s Worlds, they weren’t thinking of implementing this change (if they had, I would think they would have told us about it MUCH sooner). But circumstances forced their hand to go to the 2 team alliances. Since they had already published an agenda, doing a best of three elimination round with 16 alliances would mean serious changes in both the time allotted for elimination round and/or cut down on the number of qualifying rounds. These two options were not viable in their view, nor was going to eight 2 team alliances as then only 16 teams from 90ish would make it into the eliminations and this would not have been viable in their view. So they took the path they did. Given the above, I certainly understand (if I am right).
    There are several that have expressed that they might pull out of VEX. That is certainly your prerogative. I, however, have been a part of this since before it was the FIRST VEX Challenge and I have seen what it has done for my students over the past 14 years. I hear horror stories from some of the other robotics programs as well. I will NOT be leaving VEX. I still believe that it is the best program for my school and my students. We have been a pretty successful program over the years, in terms of awards won. However, I do not judge my teams’ success on the awards that they garner, but rather on the skills that they have learned and the people they become. The teamwork, the problem-solving, the conflict resolution, the planning, the communication, the growth in leadership and most of all, the FUN that they have as well as the networking and camaraderie and friendships that they have built with each other and other teams are, to me, the most valuable things that they get.
163

I see how that makes sense, and I think it would be a valid “test.” (My issue is them “testing” at the world championship.) Just, from a statistical point of view, you technically cannot say “at the world championship, this was/wasn’t a problem compared to the old method, so we can expect the same at local competitions.” Again, I don’t mind that, but the poster was correct that it is not guaranteed to be valid.

Edit: It’s like a trial for a drug that cures cancer among mice. It usually won’t work in human trials. Mice being Worlds competitors, humans being the general population, the drug being the changes, and curing cancer being positive or negative effects. There’s a reason we do trials on mice, but it’s not guaranteed to be valid.

164

Yup. And we start with the trials on the mice, like how we’re starting with the smaller group here.
The only way to be sure will be a full implementation, but that’s no longer a test. They want to test before a full implementation. So, given their control over and past data about worlds, can anyone name a better place to test before a full implementation?

165

Lol did you just compare our worlds 2018 competitors to mice? I’d like to think that the people who probably make up the majority of VEX’s profit from EDR teams, the very same people who’ve put in the most time, effort, and dedication into their seasons deserve a much higher position than one of a lab rat.

166

But from a statistical point of view, you cannot extrapolate data of sample of mice on to a sample of humans. That was the point that was being made. Also, I have no clue what data you think they will collect. It seen to me like it will be more of a qualitative survey, or maybe quantified satisfaction on the system, which they do not have from previous years.

167

Hey now. I made the metaphor, because it was the best one available for a discussion purely about statistics. Please don’t call him out for it.

168

My apologies. I get the point of the metaphor, although I do think that logic is flawed in the fact that when laboratories test on mice, it’s with the intent to actually roll out that product to humans over time. In this case, you’d be testing out a product on other humans, not [the equivalent of] mice. That’s the big difference here, and in my opinion that’s why the idea of testing this out at worlds is absurd.

169

1 - Regarding introducing the new format at VEX Worlds with little notice:

This is not the first time that VEX has introduced something new at Worlds. This is not a comprehensive list, but to give you an idea:

  • Raspberry Pi’s for field control
  • Tablets for scoring
  • Programming Skills Challenge rule changes (Gateway)
  • On-field sizing tool
  • Fields on risers
  • PTC tests (both in inspection, and post-match)
  • 3-field divisions instead of 2
  • Various VEX U elimination bracket iterations, including a multi-tiered one with 20 teams, single elimination for seeds 13-20, various levels of byes for ranks 1-12
  • The Round Robin elimination format
  • The concept of “divisions” and “division finals” at all

Worlds is the perfect place to insert these types of changes. We have the skilled event staff prepared to manage them, the skilled teams prepared to weather them, and the opportunity for immediate first-hand feedback. We would not want to roll something out to all Event Partners without first running it at the event where VEX and the REC Foundation are, in essence, the “Event Partner” ourselves. If it’s not good enough for Worlds, why would we ask it of the 1500+ events that occur around the globe each year?

Furthermore, I will admit that many of the changes above were made without proper advance notification to teams. Increased transparency is something that is very important to many of us here, which is why we felt it worthwhile to notify teams inside of the scheduled April 5th update. This update cannot feasibly be scheduled much earlier in the season, as State/National Championships are still running through March.

2 - Regarding the concept of “luck”:

Many have commented that this system introduces an element of luck. I think that there is a key difference between “introducing luck” and “reducing the allowance for bad luck”. First, make no mistake - there is an element of luck in all forms of competitive robotics. The concept of a randomized qualification schedule, where you only play 6-12 matches, is extremely ridden with the opportunity for good or bad luck. But, because that’s the system we’re all used to, few people bat an eye at it.

A 16-alliance best-of-1 system does not introduce any new elements of luck. “Getting lucky” in the round of 16, and the quarters, and the semis, and the finals, and the Round Robin (for HS), and the Freedom Hall finals… it’s just extremely unlikely. A team cannot trip and fall their way into winning an event, just like they couldn’t before under a best-of-3 system. I have full confidence that the teams representing their divisions in the Round Robin and the Freedom Hall finals will be some of the top teams in the world.

Now, does it remove some tolerance for bad luck? Sure. But this is the case for the qualification ranking system, too. And just like the qualification ranking system doesn’t always result in the “best” robot seeding first, there is a possibility that the “best” robot may not win the event. This has always been the case, it is why things like the Skills Challenge or the Excellence Award exist, and why the Excellence Award is considered a higher honor than Tournament Champion.

3 - Regarding “VEXnet disconnects” being the primary perpetrator of bad luck:

Every time that someone says “of course VEXnet is terrible and disconnects randomly”, a VEX field tech twitches. “Random disconnects” just do not happen, especially once VEXnet 2.0 was introduced. There is always a reason. Sometimes the reason is low battery, sometimes it’s a loose physical connection, sometimes it’s a bent pin in the competition port, sometimes it’s the fact that the VEXnet Key is buried under a giant steel mechanism that looks impressive but does a great job of shielding wireless signals. Sometimes, in very specific and identifiable cases, the reason is a field issue.

Some of these root causes are more readily apparent than others, admittedly. Blink patterns are an outdated way of communicating this information, admittedly. This is what leads to the “random disconnect” sentiment. But, that is why we employ the field tech crew at Worlds. Every division has a Senior Field Tech who is a bona fide, real-life VEX engineer and two other Field Techs to assist them. There is a Software Support center near the Inspection area with more VEX staff. The sole responsibility at the entire event, for all of these people, is to help teams troubleshoot and identify these root causes.

4 - Regarding this “being done for TV”:

I am trying to keep my comments broad rather than answering specific posts, but this is one that I want to nip in the bud right away. As part of the production crew at Worlds, in addition to being on the GDC, I can assure you that we would never sacrifice team experience or competition integrity solely for the sake of television. Audience perception plays a role in game design and event structure, absolutely. But, this pertains more to things like making sure that an audience can quickly ascertain what is going on when they see a game being played for the first time. “Best-of-1” is definitely easier for an audience to understand. But, that’s a benefit that came after the decision was made, not the determining factor.

To be perfectly clear - if ESPN themselves came to us and said “you know, this would really look better if the game objects were bigger”, or “this alliance concept is really weird, can you just do 1v1 for this event?”, there would be no chance of them having that type of influence.

5 - Regarding the “overwhelmingly negative community response”:

This thread has done a great job bringing to light many of the concerns and complaints that this particular segment of the VEX community feels. This feedback is invaluable, and the heated response is a direct representation of your passion for the program. But, in addition to the sentiments being shared here, we (Dan, Paul, and myself) have also received just as many private messages in favor of the change. Beyond private messages, Paul had the opportunity to speak with a number of teams directly at the US Open this weekend, and I have had the chance to speak with a few teams here in the north Texas area.

It’s important to remember that when controversies like this arise, the people posting are going to be those who have a strong reaction, and a strong negative reaction is almost always more compelling to send than a strong positive one. Look at Amazon reviews - they will usually be biased towards 1- or 5-star. Because someone who buys a phone case and thinks it’s “okay” isn’t going to go through the effort of posting a review. My point being - one side of the story is being very well represented here, but I would be cautious to assume that this is a unanimous opinion.

170

Since so many others are giving their opinion, I will as well.

At this time with the un-reliability of the fields and electronics I think BO3 is a no brainer. I know Paul says the fields are accurate 99.5% of the time, but as an EP, Coach and Parent I have seen much lower percentage of error free matches during my years in VEX.

In fact, this past weekend at the US Open I witnessed (MS Division) many times that robots would randomly stop working. During the event the announcer was constantly telling the audience to turn of their hot spots, so he was obviously noticing the same issues I was. If hot spots are that detrimental to the connections then we need to figure out a better way to eliminate them.

Once the field electronics accuracy improves I am more than supportive of switching to a BO1 format. Until that accuracy improves, my students success and chances of a trophy could be more influenced by field issues than their talent, skills and hard work.

171

Oh, here comes the discord train coming at me lolol…
After giving thought and changing my mind countless times, I am now swaying to favor this “Best Of 1” thing. I believe that the “Best Of 1” idea will promote teams to be further prepared for every elimination match, and also be aware of every decision they make during a match, increasing the consistency of every team. After going through all of my competitions that have “Best Of 3,” I realized that I didn’t give much care nor preparation for the first match of each ladder during the elimination rounds. I seem to always have the mind-set of “If I fail, I always have another match I could do,” and I realized pretty quickly that I lose a match simply because of lack of care for the first match of an elimination round. A great amount of times with the “Best Of 3,” I lose a match the first time, to simply realize that I actually have to win TWO matches IN A ROW, in order to proceed to the next elimination round. This is one reason why I think the “Best of 1” idea is actually a good idea.

Now, here are some suggestions I would recommend, as someone favoring the “Best Of 1” Idea.
In order to make this “Best Of 1” Idea as fair as possible, I believe that VEX Worlds should allow teams to have more time to set up their robots on the field and to ensure that everything is where they need to be before every match. I’ve calculated that the “Best Of 1” matches with 16 teams will end quicker than the “Best of 3” matches with 8 teams. I would suggest that if there is a connection issue, or it is near assured that there is a connection issue, then I would think it would be best to do a rematch to ensure that the matches aren’t being lost just by a faulty connection to a computer. I would also like to suggest that time outs be 5 minutes instead of 3, or even have two time outs per alliance at worlds to ensure that teams are most prepared for the matches. I’d like to suggest as well that the World Championship should also be open to allowing all replays by users, OR do replays with their own cameras to ensure that disqualifications are legitimate.

172

@Grant Cox Thanks for your post Grant.

I have a few concerns regarding it.

I understand that there are field techs at worlds who are able to help teams address disconnection issues after the fact. I believe most teams concern is whether, in the case that a disconnection takes place that the team is NOT responsible for, the match will be replayed or not. Cases where a whole alliance disconnects at the same time are almost certainly due to field control (the chance of two robots on the same alliance disconnecting at the same time due to a loose battery or vexnet key is incredibly low). This happened in US Open F1, as well as a high school World final match last season. In neither of these cases was the match replayed. Can you confirm that you are releasing guidance to field techs and referees that matches in which field control causes disconnects at the World Championship will be replayed? I believe this would help to alleviate many teams concerns.

In addition to this, is there a foolproof way of identifying connection issues caused by field control in the VEXU competition, where we will obviously not see two robots disconnect at once? Perhaps you could consider running both teams’ robots off of the same field controller?

Thanks for your consideration,
Lucas

173

I would like to add to @Telemascope 's post that that three of my 10 qualification matches at the US Open had simultaneous disconnections, two of which were for my alliance, and none of which were replayed. Instead the refs pointed to the green driver control light on the field control and said it was our fault.

174

Let’s just make this simple. If there wasn’t such thing as issues with field connections, VEXNet keys, as well as issues with battery connections and teams are given a nice amount of time to prepare for the matches, would most people be neutral or supporting of the “Best Of 1” Idea?

175

@Grant Cox

Thank you for your response. It’s this type of back-and-forth that creates the ideal dialogue needed to properly debate a topic such as this. Allow me to likewise address a few things about these points:

  1. The first 7 things on that list have absolutely nothing to do with this topic. Those things are either minor technical changes meant to make things easier for event staff and competitors, with the exception of the Gateway rule change, which I can’t really comment on because I wasn’t a part of VRC at the time. The last 3 bullets actually pertain to a major bracket changes like this, so those are what I will focus on.

In regards to the VEX U brackets, you’re using their situation as a means to justify doing the same thing to us (and honestly, it’s not really fair to them, either). Is it okay to change a child’s footwear from sneakers to roller blades and just expect them to adapt just because their older brother is adept at wearing multiple types of footwear, sneakers and roller blades inclusive? I realize this may not be the best analogy, but surely you get the point. Not only is it questionable for you to do this to VEX U to begin with , but using an already questionable system to justify a second questionable system doesn’t really do the second system any justice. If anything, it’ll just lower its standards by association with the first questionable system.

For the implementation of a round robin elimination format, this was actually a change for the better representation of the most skilled team. With round robin, each alliance faces every other alliance in a best of 3. Because every single alliance faces each other, there is a balanced representation of who is really the best, as no arguments like “Team X and team Y never faced each other during elimination rounds, so even though team X won the event they could’ve easily been knocked out by team Y.” can be made. Meanwhile, this change to a BO1 format with even more alliances does little, if not less than the previous system, to actually determine the best team. If anything, it does that even less due to the inherent luck factor, which I’ll discuss in my response to your second point.

I’ll skip over divisions and division finals because it pretty much gets the same explanation as the above paragraph.

  1. Let’s not get too technical about what specific type of luck is introduced or reduced, because we can all agree that in the end, the net amount of luck needed in a BO3 format is less than that needed for a BO1 format, regardless of whether or not it’s “good” or “bad” luck. In regards to a randomized qualification system, people actually do “bat an eye” at it; in fact, it’s been a large discussion for a matter of years. Several more accurate systems exist, such as Swiss (used in many popular gaming tournaments) format where rounds aren’t determined at the start of the day, but are determined after each current round ends, and teams with the same W/L record are paired against each other in order to weed out the undefeated teams one by one. Because going undefeated in Swiss events requires beating every other undefeated team, it really does a great job at not only pairing the best with the best, but gives smaller, less experienced teams a chance by pairing them with other teams at their skill level. So yes, that’s also an issue, but a completely different one that the GDC has not yet addressed.

Regarding having to get lucky all through R16 to the finals being extremely unlikely, that’s not how that works. Yes, the probability of one single team having amazing luck throughout every single match is low, but what I think you’re missing here is that we’re looking at 32 teams, not just one. The probability of at least one team having good or bad luck in at least one match in each R16, quarter-, semi-, and finals matches is actually pretty high if you consider the pure amount of teams competing. This, through a butterfly effect, skews the overall results of the competition. If seed 1 had a field control error in R16 and lost, it means that seed 16 will now move on to quarter-finals, so whoever faces seed 16 will have a much easier time than they did if seed 1 had won and no bad luck was had (which is much more likely to occur in a BO3 system). The point is, things add up. Bad luck adds up. Not just on one team, but on 32 teams as a collective.

  1. Every time a ref rules that no replay is needed on a match where VEXnet disconnected randomly, every audience member, drive team member, and live stream spectator does a lot more than just twitch. Yes, there is always a reason, but the chances of those reasons being caused by both robots at the exact same time, for the exact same amount of time is so ridiculously low, I’d argue that I’d have a better chance getting through R16 to finals on luck alone. Yet, this has happened twice on camera, both in F1 of Worlds 2017 and in F1 of this year’s CREATE U.S. Open. Please don’t pin situations like this on battery voltage or loose pins, because if it happens to both teams on one alliance, clearly it was something with the field control itself, and it’s these situations that we’re talking about.

Thank you for providing a field tech crew at Worlds, but clearly that didn’t work so well last year as the aforementioned finals match was not replayed even after something as blatantly obvious as that occurred. I’m not sure if the field techs were involved in that so much as the refs, but there are some very clear issues that need to be fixed here if this point is to actually be valid.

  1. I don’t really take any issue with this, so thank you for the assurance that your players are your number one concern. Hopefully this comes true not just in words, but in practice.

  2. Although I don’t really believe that there have been over 150 positive messages sent about this change as this thread has received replies in dissent of the change, I’ll give you the benefit of the doubt on that one. I’d just like to explain how opinions like these given through the forums are actually the most accurate opinions you’ll ever receive:

In person, people like to appeal to their audience. They like to know that they’re being listened to and they want to please who they’re talking to. I notice you didn’t mention the opinions of the people you and Paul talked to at the Open and in Texas, but let’s just assume they weren’t all negative. Those opinions were given with the bias of wanting to please a senior representative of VEX robotics. Obviously they aren’t going to say much negative, as they’re talking to someone extremely important! On the flip side, here on the forums, people can use the virtual mask of the Internet to provide the most honest opinion possible. Sure, I may get to meet you guys at Worlds, but right now, I don’t have any obligation to please you with my speech. This is why I can be as honest as possible, because there is no bias towards an elder here. It’s just pure, candid, honest speech. I can say the same for most of the other posts on this thread, or on any of the other countless threads that all echo our same message.

So, much like you say that we should be careful in assuming our opinion is unanimous (which we don’t), I urge you to be careful in assuming the opinions you’ve received in person are actually the opinions they hold when they aren’t speaking to someone as important, and are a little more comfortable sharing their real opinions.

Thank you for reading this (if you do), and I hope this echoes some of our concerns as a community towards these points. Hopefully our civil discussion can continue on in a cohesive fashion as it has so far.

176

@[TVA]Connor I would be less against it, but still against it as I believe that a single 2:00 match is not the best way to determine which set of robots is better.

177

GUYS, I HAVE AN EASY SOLUTION
For “Best Of 1,” since 3 minute time outs will probably be used by every team since teams know that they need to be prepared, think about this:
Why don’t we remove the 3 minute time outs entirely, at worlds eliminations, and instead introduce a Challenge Match where the higher seeded alliance in a branch is allowed to do ONE rematch during eliminations if they’ve lost to see if they are actually better than the other alliance. If the alliance that provokes a Challenge Match wins, they will be allowed to do a tie breaker match to be able to get back into the branch.
I think that this Challenge Match should only be allowed to be used by the higher seed only to ensure that there is efficiency on time as well.
By the way, I think the analogy of the “Challenge Match” should be called the “Challenge Match” because the higher seed is challenging the loss of their previous match.

Now, If you believe this is a bad idea, this also benefits the lower seeds as well. If I were in 16th seed, and I’m against the 1st seed. I win against them once, and then I win against the Challenge Match as well, I would become the higher seed for the remainder of the eliminations and I would be allowed to do one Challenge Match myself. This benefits everyone, and yet it is efficient on time too.

178

Believe me, it almost certainly isn’t.

179

What is it then? I’ve been witnessing these simultaneous disconnects for four years now and am still baffled as to what causes them. The field control is the only common factor that I see every time an alliance simultaneously disconnects.

180

I don’t want to either derail this thread or get into a deep technical discussion here, but take a look at this;