Community Response to Best of 1

VEX V5 General Discussion
#1

This post was ghost written by Anomaly.

“You claim best of one is so universally hated. My data and the feedback we have gotten says otherwise, but it doesn’t matter. Best of one will allow more teams to play at local events. It will also allow the time to run a tournament to be reduced. “

This was part of Paul Copioli’s email response to me about best of 1. Yes, more teams in eliminations at large events is good, and yes, less time per event is also good. Does our collective opinion matter? I hope so. But what about Paul’s alleged “data and feedback.”

To clear up any misconceptions, I attempted to survey the 192 high school teams that made elims at worlds. So far, I’ve reached 134 of these team either directly or through a friend from their region. While I didn’t survey all 20,000 teams in existence, these top competitors are a pretty big group and they constitute some of the most dedicated and excellent teams. They’re also the only 192 high school teams that have experienced best of 1 elimination firsthand. The results for the survey can be found here.

If you think there was a problem with the survey, please comment below and I’ll fix whatever you need me to. Also, if your team has not participated in the survey but wishes to do so, please comment below with your first name, team number, and team preference. If you’re a mentor, make sure you talk to your students before you post!! This survey is asking for the competitor’s mindset, not the coaches.

We already knew that best of 1 added an element of random chance and made disqualifications absolutely devastating. We knew that it motivated entire organizations to pull out from worlds because it wasn’t worth the expenses. We knew that of the 72 9th -16th seeded alliances in middle school, high school, and Vex U, literally 0 of them won their divisions.

However, now we have the knowledge that 87.9% of teams with a preference of any kind prefer best of 3. Even if every single team that hasn’t voted yet voted for best of 1, best of 3 would still be more popular. There’s been a lot of negative feedback about best of 1, but it’s been mostly anecdotal. Hopefully the plain information that best of 3 is preferred by an overwhelming majority of competitors who have actually experienced it will finally persuade Vex. Vex, please reverse this change and bring back the elimination system your community likes!

This post was ghost written by Anomaly.

#2

i think the data and feedback are from the same source as the 99.5% consistent vex keys, bring bo3 back por favor

#3

What was your survey itself? The specific questions and methods are an important thing to explain when presenting surveys. Just Bo1 v. Bo3 question? If so, you have highly tainted your survey because that is examining only one part of a number of pieces tied together. By ignoring all those other pieces, you lose information about how those things weigh into preferences. For example, if you just ask me Bo1 v. Bo3, I’d say Bo3. But if you ask me about those attached to alliances of 2, I’d say Bo1.

#4

That isn’t really a fair question either. The best case would be something like.(okay I would believe this is only a better and not “best” case) Rank the following in order by preference

Bo1 8 alliances of 3
Bo1 8 alliances of 2
Bo3 8 alliances of 3
Bo3 8 alliances of 2

Bo1 16 alliances of 3
Bo1 16 alliances of 2
Bo3 16 alliances of 3 (With note you would have less qualification matches)
Bo3 16 alliances of 2 (With note you would have less qualification matches)

These are all valid simple systems. There are then of course that slightly more complicated suggestions with Byes etc. @callen 's post is a false dichotomy.

It isnt necessarily an invalid survey. It tells VEX that they should be looking in the Bo3 direction and be choosing from options that include that because ~90% of teams want it. From there they can weigh the pros and cons of implementations of Bo3.

#5

If you go to the Nothing But Net year which has Best of 3, you can see that ALL of the number one seeds make it to semi-finals (Prove me otherwise if I am wrong), and MORE THAN HALF of those won the divisions. I am pretty certain that RECF or whoever looks through this, looked mainly at the StarStruck year, where it is a back and forth action game that the game is almost like rolling a dice itself. Yes, you can consider Turning Point to be a back and forth action game. But when you consider a game that is based on back and forth like StarStruck, then using the results of a score-only game like In The Zone, I can very much say that the statistics that Paul said has a chance to be illegitimate. I can also say that there could also be a possibility that you could even see greater improvements by keeping Best Of 3 when considering In The Zone being similar to Nothing But Net as to how the similar the scoring is.

#6

BO1 for everything and BO3 for semifinals and beyond.

1 Like
#7

I can say similarly the same thing about the Skyrise Game that has Best of 3 as well, where ALL of the rank 1 seeds made it to semi-finals.
I can also say similarly the same thing in the year before where 4 out of 5 rank 1 seeds made it past semi-finals too.
I can also say similarly the same thing during the 2013 world championship where ALL of the rank 1 seeds made it to the finals in their eliminations.
All of these tournaments had Best Of 3 did they not?

#8

connor is right- first vex makes bo1 a thing then spam makes rr and 1010 and 6627 win both semis and finals on a dq. bo3 is just less random and more fair

#9

Being part of the category that decided not to travel to worlds anymore due to monetary reasons, I can safely say that the BO1 decision was pretty much what killed even the faintest possibility of attending. Whatever motivation we had left to find a way to go was gone the moment we heard about these changes. Why bend over backwards to attend an event that punishes the very values Vex is supposed to promote inside of our program?

#10

Somehow you nearly completely misread a good portion of what I wrote, though you got the gist of it. I never proposed a question for a survey at all. There is absolutely no false dichotomy in what I wrote. I provided an example, and even wrote “example,” of how two different questions could elicit opposite responses from the same person to demonstrate how this question can skew responses relative to what the post replies to. I could have done something completely different, such as “Do you prefer ketchup or mustard?” to figure out which people would prefer on their ham and cheese sandwiches without asking about ham and cheese sandwiches.

You’re right that it isn’t necessarily an invalid survey. It shows that this group of teams prefers Bo3 when no other factors are considered.

The problem is in how the survey was planned and is being presented relative to that:

Are Paul’s data and feedback about Bo3 when no other factors are considered? Or are other factors considered as well, such as the time and number of teams he mentioned? How about issues of teams throwing matches to drop from a low 2nd seed to a high 3rd seed? How about teams feeling it’s unfair that a low-ranked team gets to win a tournament without playing in any elimination matches while they performed far better and didn’t get as far?

On top of differences in data and feedback explicitly being used to counter claims about other data and feedback, by looking at just the top teams this Bo1 v. Bo3 survey ignores constituents with issues that may be fairly irrelevant to those top teams while being highly relevant to far more lower-ranking teams (like issues surrounding the 3rd alliance member at local and regional competitions). You cannot take such a highly non-random sample of a population and honestly have any expectation that the responses will statistically be representative of the population itself. For example, let’s say we want to see how the world as a whole feeds about the USA and so we poll 1000 random ISIS members - does that sound like it would be truly representative of the whole world?

.

I have no problem with people preferring Bo3. In the absence of other issues, I would prefer it myself. I’ve also proposed some other ideas, wondering if elements of Bo3 could be maintained in shorter timespans without having Bo3 so that other things could be adjusted.

What I do have a problem with is someone using such a skewed survey to make claims such as “now we have the knowledge that 87.9% of teams with a preference of any kind prefer best of 3” and being used explicitly to refute a different set of data and feedback. That is either statistics being wielded and mangled by someone without much understanding of statistics or it is scientific dishonesty.

#11

Good job collecting this data.

I thought your data collection was fair and not leading and the sample size was appropriate.

I doubt you will get the powers that be to back pedal on a decision of this size, but at least you are attempting to understand what the voice of the competitors are.

My suggestion would be this, before you publish this data I would take the time to graph it out and provide provide the question you asked. I’d also shy away from publishing what every team’s opinion is, and instead simply provide how many teams chose each option.

I’m usually a stickler on how survey’s are conducted. But OP made it clear why he chose his sample size and his logic is sound. Those are the only people in the World to actually have hands on experience with that system, and having seen other “surveys” published on this site, this is by far one of the most thought out and best conducted I’ve seen.

Show me where OP said he was trying to survey the whole world. It was simply stated as a survey of the teams that have actually participated in bo1 style elimination.

Having been one of the teams reached out to, I know his question was fair and not leading, He simply asked if we preferred bo1, bo3, or were undecided. You cannot be any less leading without being unable to ask the question.

I highly doubt the game committee has actual data on feedback, or care. They’ve made their choice and aren’t going to back down. It’s admirable a student tried to understand what the actual voice of the competitors was.

#12

Yes, it was made clear. As I said, there is certain validity. But then look how it was represented: “now we have the knowledge that 87.9% of teams with a preference of any kind prefer best of 3.” He’s saying this is representative of “teams with a preference of any kind.” That would be any team anywhere in the world that might care. I’m not saying he planned to survey the whole world, I’m saying the sample is being specifically stated as being representative of “teams with a preference of any kind,” which would be far, far more reaching than the teams he surveyed. But that’s not how the sample was determined.

Read my above posts more carefully. Look at what this is specifically being used to refute. This is a very leading question related to what he is using it to refute. If all he were doing were stating that in the absence of other factors these teams prefer Bo3, it wouldn’t be leading. But that’s not what he’s stating, and for what he’s stating it is very leading. And if you were one who was surveyed and don’t see that, that essentially proves the point. You were being asked to provide data to support or refute Bo1 v. Bo3, timing issues, and more while only being asked about Bo1 v. Bo3.

#13

This is just nit picky.

I did, I do not care to again.

I disagree, but I doubt you’ll ever concede that so keep on believing what you will.

#14

No, it’s not. It’s the crux of the entire point of the OP!!! Either he thinks it is representative of the world, meaning he doesn’t understand sampling; or he does understand sampling and knows it isn’t representative of the world and claims it is. Either way, it does not support his statement.

Nothing I can do if you misread and don’t care to reread.

#15

Your word soup is difficult to parse out the first time, I don’t feel like going through it with a fine tooth comb a second time. Present you arguments better and more concisely.

I think OP made it clear that he had a understanding on what the surveyed teams preferred. His data does not necessarily relate directly to what the rest of the world may prefer, but it’s absolutely fair to speculate based on his sample size.

#16

I agree with both @callen and @TheColdedge in the following respects. The data collection was fair and the sample size was adequate. As a teacher of AP Statistics, I would have a bit of a problem with the sample being representative of the whole VEX population. However, it is not surprising that a vast majority of teams would choose BO3 over BO1. If that was the ONLY question, the results are valid and predictable. It is important to understand what prompted the decision. To me a much more valid survey would be to give teams the background information and then give them various options as @tabor473 suggested. In my opinion, without understanding the reasons for the decision, BO3 will always win out over best BO1. It would be much more valid to see, if after being presented with RECF’s reasoning for making the decision, what a survey giving various reasonable options might reveal.
This is my take on the reasons for the decision. The BO1 is a consequence of the move to 2-team alliances. The move to 2-team alliances alone is a move that I think many would support, if the question was, “Do you prefer 2-team or 3-team alliances?” The move to 2-team alliances was, in some way, meant to deal with an issue that seems to occur in some places and that is the “tanking/throwing” of qualifying matches by teams to put themselves in the position to be the 3rd team in a top alliance rather than the alliance captain or 2nd robot of a low seeded alliance. This practice, in my opinion, is certainly not in fitting with the values of the RECF. Once the decision to go to 2-team alliances was made, then thought had to be given to the elimination rounds. If there were to be only 8 2-team alliances, then only 16 teams rather than 24 are advancing to eliminations. Given that tournament brackets work best in even multiples of 8, the next size up that made sense was to go to 16 2-team alliances. However, going BO3 would take a LONG time and would result in events either ending much later of severely limiting the number of qualifying rounds. Neither of those options were palatable. Therefore, the move to BO1. It takes approximately the same number of matches as 8 3-team alliances running BO3.
I have NO problem with a survey. I have no problem with this survey, as long as we recognize its limitations. I do think a different survey would give us much more usable information.

#17

Ya I jumped the gun a bit. I am honestly so impressed with the effort these students put in I felt the need to defend it. VEX has stated their opinion that Bo1 is actually fine and that no one has a problem with it, it seems that the vast majority of teams that have experienced it do. We almost certainly don’t have a consensus on what people do want, its too complicated but we have a place to start.

I personally do think asking those it affected is a good sample to collect. Not a single other team has had the opportunity to experience this type of elimination round so they are uniquely qualified.

And finally, I have talked with the people performing the survey in previous weeks. I would like to publicly mention before it comes up that I believe the data collected is valid. (Just dont want someone coming along and saying its made up)

#18

That’s not my statement, though @536Mentor and I seem to be in total agreement (sample not necessarily representative of whole community, single question with a predictable outcome asked and applied to a situation with multiple questions). Not that I want to waste time doing it, but I wonder how this same group would feel if I did a survey of all of the teams at a single local competition and asked only something like, “Would you prefer the current number of 3-team alliances or more 2-team alliances in eliminations, or no preference?” and similarly tried to use this to support or refute the broader picture. This would be leaving out the other issues (like Bo1 v. Bo3) while I would be applying the answers to those issues.

I don’t doubt that. As I’ve said, I believe the data are valid (collected properly and have meaning), just that they don’t come close to applying as stated nor to show what they’re stated as showing (refuting Paul’s data and feedback and broadly applicable to “teams with a preference of any kind”).

#19

@Unionjackjz1
As former captain of 8110x The ribbits for NBN-ITZ we played in eliminations throughout a majority of our events and all of them this season.by seeing the benefits of both I believe we lost in the semifinals at worlds because of the best of 1. Our opponents outs cored us 64-4 after autonomous and we made up ground only losing the match 120-129. In that case we would of preferred a best of 3 but I also look back at our state championship where we as the first seed alliance captain played the 8th seed in quarterfinals. Let’s just say it was not a clean match… our alliance disconnected during auton and we had to 1v2 the 8th seed. This was a situation where best of 3 also gave us support and was less stressful. But I can also look at some of the matches we played and it could have gine for the other teams in world elims. I think the Best of 1 from a spectator standpoint is better because of everything being in the line and it promotes consistency in robot design. So overall I like the Best of 1 because of the atmosphere it brings to the competition

#20

Change is not something that many of us like so asking experienced Vex teams who have been successful with BO3 for several seasons if they would want to try something new would probably not receive much support. Especially since we have not tried it for a season.

BO1 will be a complete change in how teams compete and prepare for elimination rounds. Several organizations (including mine) have “that one team” that wins many of the tournaments - this might change that. I think that BO1 will make tournaments more interesting with the potential upsets from underdogs.

I am simply offering a different perspective.