Concerns With Rule T1b

VEX V5 Robotics Competition VRC > Tipping Point (21/22)
62

Thank you - I was looking at saying something similar.

EPs are focused on providing best overall experience for all teams. We are not positioned to “overturn bad calls”. Moreover, having EPs focused on that one match to assure that one team to advance to Worlds is a disservice to all teams at the event.

EPs provide the space and recruit volunteers for events. This video replay mandate will lose more EPs when we are faced with shortage of event venues.

As for getting to Worlds - build the best robot and have best skills run, or earn Excellence at your championship, or maybe win Online Challenge. I reject that “one bad call” hypothesis totally blew all opportunities for the team to advance to Worlds on merit.

Apologies if I appear antagonistic - but let’s be realistic there is a long road ahead before you can offer quality video replays at all event venues.

63

As a competitor, I only cared about one thing, and that was winning the tournament (not excellence or skills, being the champion of the tournament). In the process of being shortsightedly and narrow-mindedly focused on the competition, I learned the programming skills necessary to get an internship which eventually lead to a good software development job. I don’t understand this seeming antagonism towards people who are competing in a competition and want to treat is as a competition.

I continue to volunteer in this program because the competition taught me valuable skills, and I want to pay it forward, but it is fundamentally a competition, and I continue to treat it as such.

I agree that EPs should have nothing to do with overturning calls, this would be a ref decision. But as an EP, while I want to ensure all teams have a good time, I also want to ensure the teams that have worked the hardest and built the best robots are fairly rewarded for that, and I would be happy letting elims run a little longer if it meant my referees could be more confident in their decisions.

In my opinion, it should be optional at local events (the stakes are lower) and possibly mandatory at a states event. A state/regional qualifier should really already be livestreamed anyway. Sure this will mean some events are seen as “better” because they have a review setup, but events are already “better” because you know the EP will run it better, or have better refs, or have more qual matches. If there’s a shortage of event venues, all the events will get filled up anyway, so what does it matter?

I’m not a competitor this season, so actual competitors please correct me if I’m wrong, but I believe the meta for designing a skills robot is different than the meta for designing a competition robot. In any case, depending on the region skills won’t necessarily qualify you for worlds anyway. And having been a judge and a competitor, I know that it’s nowhere near as simple as “build the best robot and you’ll for sure win excellence”.

I might be biased from my background when I competed, but I really never cared about judged awards or skills. The competition, in my view, was the tournament, and the winners were the teams that won finals. That’s the only thing I was playing for. And I don’t think it’s wrong to be focused just on that.

I want to make it clear I’m not actually sure how well allowing refs to review video would work. Maybe it would be terrible, it’s entirely possible I’m wrong and it would result in a ton more arguing and increased uncertainty. My guess is it would make it easier for the refs to make a few calls in elims, but make the cycle time a bit slower, which I think is a fair tradeoff. To the extent I’m personally in favor of anything, it’s “run a few trial events with this allowed and see how it goes”.

64

Not necessarily, if you don’t get a worlds spot, your season ends. If you get a worlds spot, you get to do robotics for another 2 months.

65

Our teams routinely do not get Worlds spots and yet manage to keep club and interest going year round. this is both at our middle schools and high school.

During Change Up - there was no way we had any opportunity to have any sense of a normal season, yet club kept on virtual, hybrid, and explored new interests - including robotic gardening (farmbot), flying robots (RAD drones!), and traditional robot builds and VEXcode VR (thank you VEX!!). We know this season may be shutdown again (thank you Omicron) and creates a barrier to Worlds for us. We roll with it.

@9MotorGang in the grand scheme of things - this viewpoint “if you don’t get a worlds spot, your season ends” is extremely narrow and does not apply to 96% of teams across the globe. I wish you could see be open to more than just Worlds as your “season” …

I get it is disappointing when decisions don’t go your way - literally we’ve been there done that as an organization. There is so much more to robotics than just Worlds. I would focus on how to make local championships the high light of competitive season - celebrate amongst your peers around you year round.

66

Did you get the internship/software job because you went to Worlds, or did you get them because you wanted to win competitions and go to Worlds. Those two are very different.

While qualifying for World’s (or winning World’s) is a great goal for many competitors, not achieving that goal because of a bad call is not some egregious wrong that must be righted or avoided.

I am sure your sentiment:

Is common amongst many (probably a majority) of competitors. It’s a value statement and can’t be “right” or “wrong”. I’ll make an assumption here, though, and say that virtually any team believing this is not likely to turn down a World’s spot because they won Design or Skills but weren’t State Tournament Champions. Which leads me to believe teams really value the status (and experience) of Worlds, regardless of how it occurs. Perhaps I’m wrong.

What those of us gray-beards are suggesting, is to try to take a step back and evaluate what one’s values truly are, perhaps with a longer time horizon than what most young people (our former selves included) typically use.

67

That’s certainly valid. When I was competing and what I instill to this day is dominating and winning in everything. Excellence, Skills, the tournament, etc. I’m not sure where this shift in just focusing on just the tournament and the competition happened, but it saddens me a little bit. There’s so much to be gained in this program beyond just the competition field.

I say this as an ultra competitive person, and anyone of my students I coach or have coached before know how competitive I can be and where my focuses lie during the season.

This is not antagonism against competitors, but this echoes my sentiment:

What those of us gray-beards are suggesting, is to try to take a step back and evaluate what one’s values truly are, perhaps with a longer time horizon than what most young people (our former selves included) typically use.

68

This is certainly a valid point. But with that said, my desire to win the competition was based on a belief that tournaments I lost were a result of failures of my robot, because the rules back then were way simpler and elims were BO3 so a single bad call wasn’t the end of the season. I don’t know if I would work as hard if I was competing in the current system, because there really does seem to be a lot more randomness and judgement calls.

Please don’t take the rest of what I’m about to say as a response specifically to you, but I’ve been noticing that a lot of the EPs/mentors on the forum tend to respond to students raising concerns about the quality/integrity of the competition (e.g BO1, white screens, possibility of cheating) with a mixture of a few responses

  • You need to take a longer view, VEX isn’t just about the competition, it’s about education
  • In life sometimes things just happen, this is a learning opportunity that life isn’t fair
  • If you just focus more on excellence and other judged awards, then you can win that way
  • Yes, but even if we do X, Y will still happen, or it’ll only fix it some of the time

And what it seems to boil down to is there’s a fundamental disconnect

The EPs/mentors/etc (rightly) see this an educational tool and want to teach engineering as well as life lessons, the outcome of the tournament doesn’t really matter as long as the competitors are learning

The competitors (rightly) see this as a competition and want to win or lose on their own merits, and find it unfair when unfair things happen. They’ll learn what they need to get the outcome of the tournament they want (as long as they believe that will work)

Both sides achieve their goals as a byproduct of the goals of the other. But, while students will learn things like the adults want naturally as a byproduct of competition, the adults running the system (in my opinion) have the obligation to consider the quality of competition to ensure the cycle continues.

I don’t think video replays will singlehandedly make or break the integrity of competition, but I do feel like a lot of the general sentiment by the EPs/mentors leans in the direction of “the integrity of the competition is irrelevant, just keep learning” which does concern me

69

Perception is reality.

The RECF-adults made changes and now allow replays for white-screens.

In fairness, the RECF-adults did listen (partially) and have changed to allow for BO3 at World’s qualifying events (Signature, Regionals, etc.). I’d done research and analysis on the BO3 question and found that, in aggregate, it didn’t alter very many outcomes. Perception is reality though, and competitors will say “But that coulda been me coming back from a white-screen to win back-to-back”. A perspective of the big picture versus a view of a specific incident.

One of the ways I believe I fill my role as a mentor is in providing a different viewpoint to that of the students. I humble and introspective enough to consider that the way I communicate that can improve.

This comment upsets me, and I understand you are not directing it at any one specifically.

All the EPs, all the mentors, all the refs that I’ve ever interacted with do their utmost to host competitions that are environments for teams to have fun and compete on a level playing field, so that each team’s outcome is determined by their own efforts and skills to the largest degree possible.

I’ve done other analysis comparing win-rates of seeds during VRC elimination rounds. These have not changed markedly over the years. In fact, VRC seeding is more accurate than the NCAA Men’s Basketball seeding. But perception is reality - “I was a 1 seed that lost to a 16 seed once and that should (almost) never happen”. A perspective of the big picture versus a view of a specific incident.

At the end of the day, the adults are putting on a competition format meant to foster and reward an engineering mindset in students. Part of an engineering mindset, IMNSHO, is solving the right problems, and understanding that solutions have costs, sometimes hidden costs.

If the larger issue that “instant replay” seeks to address is “VRC’s competitors believe that unfair outcomes occur too often”, then lets tackle that elephant in the room with an engineering mindset.

70

What’s funny about this is that I still know some adults who are very upset about how our season ended in 2018, including our team’s mentor who saw how hard we worked all year. Long story short was that was the year they decided AT WORLDS to change eliminations to BO1. I laughed to myself immediately when I read that, and said to my teammate, “Yeah, we’re definitely gonna get screwed by this somehow, aren’t we?” Suffice it to say that I feel we did.

I don’t really think about it much anymore (this post excluded) but it’s funny to me that many adults do! Specifically, I met the wife of my former teammate (this teammate being 24) and she told me how much he had told her about me, and how he said we got screwed and deserved better at Worlds, and how he shows her all the videos. Mind you he was not even competing when this happened, he was just very passionate about it so he made the trip out to Worlds to hang out and scout for us! So there are a lot of people who care about competitors getting what they consider a “fair shake,” and it’s just not the competitors themselves. And these memories really last a lifetime! Some of us are driven and take pride in being the very best, if we can be.

71

thank you for posting this - I, too, am discouraged by this post.

@AlecMiller when you say “Please don’t take the rest of what I’m about to say as a response specifically to you, but I’ve been noticing that a lot of the EPs/mentors on the forum” literally that is a specific response to all the EPs who are on the forum.

I am an EP on the forum
I am a coach on the forum
I am a mentor on the forum
I am a parent on the forum
I am a volunteer on the forum

My guess is you feel that EPs / Mentors / Adults are not empathetic to your worries/desires. You could not be further from the truth.

As @Mentor_355U noted - a lot of change has happened because we do listen. If we appear to be dismissive, perhaps it is because there is a lot of understanding the operational aspects that might not be appreciated by competitors in general. For example, how to manage changing events in response to pandemic on the fly to give all opportunities for all team.

72

Apologies, concluding what I said with that didn’t do a great job of summarizing the point I was trying to make, which is simply that I’ve often seen responses to (in my view) reasonable criticisms of competition conditions met with the kind of responses I described, and it feels like it results from a difference in perspective between people who approach it as, at the most fundamental level, VRC-as-education versus VRC-as-competition. Just as an example of how that difference in opinion can manifest

I’d done research and analysis on the BO3 question and found that, in aggregate, it didn’t alter very many outcomes.

Absolutely, I’m sure this is the case. However, some (including myself) would argue that it’s still worth the tradeoff, because even if it wouldn’t alter many, it would alter some outcomes. But if you’re more concerned with VRC-as-education, BO1 means more teams in elims, which means more teams get a chance to participate.

But yes. Poor choice of how to conclude my thoughts. Apologies.

73

Thanks @AlecMiller - I appreciate your sincerity.

Apologies for being going a bit off the rails here in a BO3 tangent, but I do feel that part of the real story for the change had to do with the 3-team alliance part of BO3, where teams were rumored to “tank” qualification matches in order to be a “sleeper” pick for a high-seed alliance. Whether that actually happened or not, is nearly impossible to say, but there was that perception. And the elimination of 3-team elimination alliances seems to be one that would then increase the integrity of the competition. I think everyone can agree the execution of this change was poor

One can reasonably argue the merits of 8 2-Team Alliance BO3 versus 16 2-Team Alliance BO1 elimination formats and the sort-of hybrid 16 2-Team BO1-sometimes-BO3 that we have now.

Though I’m not sure I agree with your premise that there’s some sort of dichotomy between “VRC-as-competition” and “VRC-as-education” that exists between adults and participants.

74

Oh dang! Thanks for that, kind of important. And yeah, Referees are (supposed) to be the experts. However sometimes Referees EP Events and need to find Referees for their events, and since everyone else has [you] to referee, no one else bothers to put the time into learning it.

@sazrocks That is what I expected to happen. The time taken to weigh what happened and talk to other judges would basically be gone, because with someone presenting the referee evidence, they can either not change anything (no significant extra time), or get the call right.

@OscarMNOVA12 More investment gives more camera angles gives less ambiguity. Therefore shouldn’t any be better than none? It is easy to make obvious calls when under review. If you missed an easy call, then the video review was worth it. If you missed a difficult call that would have required lots of technology to catch it… you still did well, and probably didn’t stand a chance to see how it went. You also don’t need to watch everything all the time. All you need to do is, when asked, take a second look. If a ring is under a platform that causes it not to come down all the way, and you aren’t sure who put it there, wouldn’t you like to know? If a team loses rings after a defensive bout with an opponent, was it an action they took or the opponent that caused the rings to become descored? If a disabled robot gets pushed into their own corner where there are two goals at rest, wouldn’t you like to just review if was actual hoarding or if they were forced into it. I even had a game piece bounce off of a player and their match controller cable as it fell over during the last 30 seconds, not touch the ground, and then slide down the platform back into the arena. It should have been removed from play, but I did not catch it in time, and was unable to review footage.

@SOLAR_YT I only wanted the cameras to be placed by the event staff, and operated by the event staff. Teams do not get to review the footage along side the referee, but they are free to review it on the stream. Additionally, a bad angle cannot determine what has happened if it can’t see well. It can only provide insufficient evidence. If it doesn’t provide information to make a ruling, it cannot somehow cause a ref to overturn their ruling the wrong way.

@Mentor_355U There need be no opinions shared. Either the Referee saw enough evidence to overturn, or they did not, and the specific invocation of rules at hand. Any more time spent nitpicking and arguing is a fruitless waste of time. If the Refs don’t know the rules to begin with, video isn’t going to help that. It should take little to no time, and can be prepared by another party while the ref continues the competition.

As a referee, I feel that I often do not have enough information to make match affecting rulings that influence Excellence and Tournament Champion winners. I would say this happens once out of every 50-100 matches I ref. However many of these matches are blowouts. During close matches, I would say that I have insufficient information every 20 or so matches, and could use a video review. There are better referees than me. However they are busy reffing other events. I can study the rule book for weeks, but unless I am both skilled and practiced, I cannot see everything I need to make all the right calls in the moment. IMPO, a well described video review would increase the quality of officiating at most events in such a way that it outweighs the frustration caused by a few minutes of extra delay.

In summary, the infrastructure is there, the time commitment is small, the personnel requirements is small. Why not give us the option?

75

The number one reason I am involved with Vex / RECF is that they work very hard to provide a level competition model regardless of financial, gender, or any other social classification. As an EP I need to maintain consistency and the subjective rules are discussed at the drivers meeting. The rules may change during the season but as long as the referees are consistent for that event then every team has an equal chance.

If one team has the resources for video replay where other teams may not then we have inconsistency. Team X with video replay resources should not have an advantage over Team Y. That leads to the event partner responsible for the replay and I would argue that in fairness all events would need to have video replay. Teams from different regions should have all of the same resources and that is not feasible right now.

Governing bodies (NFL, FIFA, NCAA, RECF) are required to make decisions for everyone involved. If the error rate is less than 2% of matches, I don’t know why we would invest into this concern over so many other opportunities to expand or improve student experience.

76

Propose it to the powers that be. Run a sanctioned experiment with it. But my prediction is that it causes as many or more problems as it solves.

I’ve recently been reminiscing the old PERL-ism “Some people, when confronted with a problem, think ‘I know, I’ll use regular expressions.’ Now they have two problems.”

EDIT: Having been an LRT ref last year, I can say that was not a great experience. “Hey team, go take the camera and walk it up to the robot so I can see if it’s touching the ball”…good times.

77

I cannot agree with this more.

78

I would like to say… what is the downside to such a rule addition added?

The key word is “may” where the refs may review footage. That does not mean there has to be footage to be reviewed. The concept is to allow competitions with such capabilities for replays to be able to utilize the capability to better improve the quality of the event. If an event does not have such capabilities for replay then the rule does not need to be in play for the event. There is absolutely no downside to adding such a rule.

For some, adding replay systems could be expensive. But for others it can be figured out for cheap. It really depends on the situation. Why not allow something that can better improve the quality of the events that wish to so?

79

I would also advise that the rules read that the video footage must be “Official video footage” so that everyone knows its whatever the EP deems ahead of time, whether it is streaming video feed, or a select volunteer in the front row recording matches on their phones. That way random audience members and teams don’t feel compelled to provide their own video evidence, bogging things down.

Also, it should be noted that if teams need to request official reviews, they do so before leaving the box, and without consulting video or audience members. Since people not in the Drive Team Box cannot advise the Drive Team, they should not have a say in requesting reviews. If teams consult with individuals not on the drive team, then they forfeit the right to protest the findings.

Putting it all together, … "The Head Referee may review official video to determine a score or ruling. This can be done immediately after the match or at a later time when the video is ready for review and when the Head Referee has time to reference materials or resources to help with the decision. "

It might be worth defining official video. That being said, the current structure of T1 and T2 make it so that adjusting the rules fits right in.

In response to this, I don’t see why we wouldn’t take every simple opportunity to expand and improve student experience. I cannot personally remember a competition that would not have benefited from this, and I have been a referee since Starstruck. I feel that the actual number of matches I would want video review for is closer to 3 or 4 percent, but in reality it is much higher in elimination rounds, where I feel it matters most. Those rounds tend to have more interactions per minute and faster robots capable of imparting more energy onto game pieces so they wind up where they shouldn’t. They also know the rules and walk the line. If they can draw a DQable foul and win the match outright, why not.

@Mentor_355U The problem I am trying to solve with this rule is just a lack of information and I would love the opportunity to test this procedure out and see how it goes.

This is how I imagine it going:

  • With 40 seconds left a Red robot drives to tilt the Blue Platform and plows several rings underneath the platform. However it goes unnoticed, as the robot obscured the referee’s view. Blue alliance is unable to drive onto the ramp because of a ring that is lodged underneath their platform. They try to clear it with 10 seconds remaining, and then attempt to plat, with time expiring.
  • I determine it to be match affecting. However I score the match up and have the Scorekeepers make a note that there was a ring under blue alliances platform. I announce that the current ruling is that no rules were broken and I am going to review it. "Please verify your scores and I will review the video.
  • I walk past the streaming table and ask them to review the video and find when rings were first pushed underneath.
  • After refereeing the next match, I walk back by the streaming table and review the video they have prepared. It seems that there were no rings, and then after the Red Robot drives over, there are now rings. After this interaction no other robot touches those rings.
  • I determine this to be a violation of SG3.c and Disqualify the offending team as it seems that gameplay was interfered with.

The DQ’d team feels burned (assuming they didn’t plow the rings as part of a defensive strategy). However they did engage in a risky action and didn’t ensure they were not in violation.

All the scenarios I can think of: Determining who caused rings to come off of an Alliance Goal, determining if an opponent forced a team to commit a foul (and I didn’t see the forcing happen), if I can’t tell who caused a game piece to violate SG3; they all share the fact that the referee missed something. If refs have a chance to look back and confirm what happened, they can make the right call. To me, the right call is the one that perfectly mirrors the intent of the rules.

Both Excellence and Tournament Champions depend on the referee making the right call. Someone will always get hurt when penalties are assessed, but they are designed to hurt the correct people. The game is simply the sum of its rules. Without them, this is just interpretive dance.

80

Sorry, the Excellence Award is not dependent on Referee making the right call. They do look at overall performance of the team/robot/design. The judging process is independent of outcome of single match on field.

81

As @Foster recently posted, he’s looking at whether to spend $1k on Banners or make $1k in funds available to teams.

Rather than spend resources on enabling VR for an EP’s set of tournaments (some EPs only host 1 event per year, others do multiples), perhaps putting those resources towards making more tournaments available in an area (or larger tournaments, or better funding for teams) would provide a better experience for more students that getting the call right in 2-4% matches. That’s the argument for BO3, right, is that over more matches, skill wins out over luck.

I think what many of us naysayers on VR are trying to point out is that VR isn’t as “free” as the proponents make out.