1 would increase the length of an event far more than the amount of time required to review match footage given the number of close judgement calls I’ve seen this season.
I completely agree with 2. Some rules are insanely difficult to enforce, SG3 being a great example.
For 3… we have 3 referees on a field and still miss things. We have the resources for video replay, but not the resources for more referees on a field. In some regions they might have the resources for more referees on a field, and that’s great, more power to them.
I feel as though if we want to talk about time we must understand that VEX Robotics went to BO1 for the sake of saving time.
Reviewing footage will have moments where the time is shorter. Instead of a “he said she said” sorta experience to determine if a ruling is justified or not, you can spend 10 seconds looking at footage and be like “yea thats illegal.” But if not done correctly it will add time if electronics fail to work or there’s not enough pixels on the screen. VEX Robotics is trying to make competitions equally as enchanting to people and students, but not make it too lengthy. But honestly, I feel as though replays should be at the discretion to the event to decide if they should or shouldn’t be allowed.
If competitions have the capabilities to replay video footage, then why not allow it? Replays should be encouraged in competitions, but they do not have to be required. They should be allowed, but optional. That way competitions can have the leeway to make their event the way that they wish it to be. Some competitions want the event to be memorable. The entire concept of VEX Robotics is to learn STEM in a competitive environment, yes, but the ethos is also to make memories. How cool would it to be to allow competitions to have replays, making the environment feel like an authentic sport? If a competition had a desire to make the event feel memorable, and they have the funding and sponsors to do so, then why not allow them to have replays?
When you consider the concept of worlds, only the cream of the crop top 1% (perhaps even less) make it to such a place. The only place where replays are done are at worlds. So if the concept is about engagement then why do we only restrict replays to worlds, if there may be a competition that would like to have replays to make a more memorable moment to individuals who miss out on such an event?
I also think I wasn’t quite making my point correctly. Let me also say I don’t really understand sports so I could be wrong
In VRC, the referees are unpaid volunteers, often without a lot of experience, watching a game that’s very hard to follow, and there are only 3 of them.
In most sports, the referees are paid, with at least a basic amount of training, watching a game that follows a single point of focus.
The status quo for live referees for VEX has a much higher chance of match-deciding error than does normal live sports.
In VRC, it it possible to follow a match (even if some details are missed) from a single camera capturing the entirety of the field.
In normal sports, in order to follow the match with only some details missed, you need live camera people and multiple cameras.
The barrier for any kind of useful feed in VEX is far lower.
So in sports you have a better standard of live refing, and video replays are far harder. It’s unlikely a low end video review system would yield benefits.
In VEX you have a lower standard of live refing, and video replays (even if imperfect) are far easier. It’s much more possible a video review system would yield benefits.
I also do not believe that LRTs are a good test run for video replays. The camera angles were not optimal to say the least, the spaces were not optimal. It’s much easier to set up an angle of a field that shows the entire field well and has no drivers blocking the way if you’re in a gym rather than in some random room in a school.
Also, there are circumstances where live refs would get a better view, I don’t disagree. However, it can be useful for live refs to be able to go back and get context, especially for rules like SG3. Often that context can be gathered from less than perfect video sources.
I don’t think this would be a perfect system that would ensure no bad calls are made (I watch F1, believe me, you can have all the telemetry and video angles you want and there are still gonna be some divisive calls). I just feel like not allowing basic video replay if available at the referee’s discretion is letting the perfect be the enemy of the good.
@Jon_Jack By the way, thank you so much for your efforts and time responding to this discussion. I am incredibly grateful for all the time you are setting aside to provide insight about the thoughts on the other end. I feel as though there is too much aggression in this thread and that we are taking your involvement for granted. And we are sorry about that.
I don’t see any pins in the picture you shared? I mean I guess they’re within a tile, sure, but the picture doesn’t tell you how long they’ve been there? Maybe they were driving by? Maybe they’re trying to trap the opposing robot? It’s kinda hard to definitely make that call without context…
Definitely easier if you’re in person. If you’re trying to count with video replay on just that one view, then good luck.
Honestly, you’d be surprised. I would venture to guess that we spent longer than 5 minutes per match we reviewed at LRW (find match, watch match, track scoring, deliberate scoring, rewind, deliberate, etc.). Lets say it takes 5 minutes to get robots on a field, run a match, and reset the field. If it takes you longer than 5 minutes to review video, then it’s actually quicker to just replay the match, right? That still doesn’t change the fact that either option adds time to an event.
Even if you only do this in elims, a pure BO1 bracket has 15 matches. If you have to review every match, well now you’re talking about 75 minutes of extra time - and that is assuming there’s nothing else that delays the event.
Let’s say you only do replay at a Worlds qualifying event, who wouldn’t challenge a call in a match where they’re going to be eliminated and not qualify for worlds? 16 alliances… 15 alliances getting eliminated… 15 challenges… Lots of time added to an event.
I like playing robots, but not everyone is going to be cool with staying an hour or two longer than scheduled to finish an event.
I feel like this is a pretty big assumption. Does it exist? Yes. Is it sort of accessible? Yes. However - how many events even stream video? How many events archive that video? I feel like until this is more standardized, then this statement is false. Perhaps the proliferation of standardized streaming and archived video is a better starting point. Like they say - “You have to crawl before you can walk”.
Since you’re someone who admittedly doesn’t understand sports, I will add that calls are missed all the time, even in the big leagues. Like literally career and game defining (for the referee, the player, the coach, etc) calls are missed. Those refs also have an enormous amount of training resources at their disposal. Yet, they still mess up. That’s why Twitter accounts like this exist…
Improving referee quality is something being worked on. Dan and his team at the REC have made big improvements even within the last few years. Is it perfect? No, but neither are those paid refs you talked about.
I even play in a recreational hockey league. Our referees are paid per game. They also have to go through recertification every year, even though the game doesn’t change year to year… They still make mistakes. It happens. We don’t have video replay. Has there ever been a moment after a blown call where I said to myself “I wish there was video replay”? Never.
The process of finding video, watching, deliberating, reviewing, etc. wouldn’t be all that different between an in person event and an LRT event.
Can I pose a question?
How does video replay help the REC achieve their mission of “Increased student interest and involvement in science, technology, engineering, and mathematics by engaging students in hands-on, affordable, and sustainable robotics engineering programs.”
I don’t mind. I know this is a hot topic. It was even when I started in robotics nearly 20 years ago. I’ve had a lot of time to ponder this topic. I think it’s foolish to say it’ll never happen. At some point down the road it will, but I also think there are some legit concerns that have to be overcome before it can happen.
I’ll end with this -
As a former competitor I’ve lost events because of missed calls in the finals, so I get where the desire for video replay comes from. It sucks in the moment, and feels like it’s pretty big freaking deal. In hindsight, it’s not. It’s just a moment in a high school robotics competition. Losing those events on a missed call has had absolutely no bearing on my future life, other than I got to learn first hand that people are human and can make mistakes.
I think a fundamental flaw that video replay will fix the unexperienced referee factor due complexity of game you keep bringing up is flaw. If the referee can’t make the “right call” live, what makes you think they will interpret it correctly on video replay? Teams demanding a video replay will just frustrate the volunteer more by putting them on the spot. You may find yourself with referee backing themselves into a bad mindset (human nature). I’ve see this over and over in youth officiating. Worse, is volunteer drops completely.
Also, barrier to entry for EP gets higher with more complexity to running events. Finding more gear and now training volunteers how to do video replays correctly rule wise and operationally (TM folks etc…) will freak out nervous new EPs.
As for bringing Video Replays to Worlds - well we know teams are not happy everytime entry fees are raised. Now adding complexity to every fields at Worlds is a huge deal and costly! Are we prepared for Worlds to cost $2,000, $5,000, or $10,000… or mask the true costs by raising team registration fees to $200 per team? This does not happen for free.
Pinning might be the wrong word. In this case, I was going back to the video that I took this screenshot from, and one of he robots was ramming the other robot into the corner. My point was basically that, to the extent the one robot was blocking access to the rings, it was because the other robot had just pushed it into the corner up against the rings
I don’t want to get too into the details of this one hypothetical since it’s… not really the point of either one of our arguments, but if the possession limit had been like three or 6 it’d be easy to count with video that they’re breaking the rules. If the possession limit had been ten… I honestly might not be volunteering because trying to enforce those rules would make competitions a complete nightmare.
I’m curious what exactly you were reviewing since you keep bringing up scoring. I am in complete agreement that attempting to figure out the score from a video replay would be a nightmare. I don’t really see much reason video replay would be used to check scoring. We show both teams the score before submitting and resetting the field to make sure everybody is in agreement, the saved score is the score.
Sorry, when I say “we” I mean the group of people who I run events with. Not we as in the wider VEX community.
Absolutely. I’m not saying live sports refing (or even NFL refing with super expensive video review) is perfect. Calls will be missed. Sometimes those calls will decide a game. I’m just saying that live sports, even at the high school level, have paid refs who didn’t just learn what the rules of the game are that morning.
And everything else aside, it’s a lot easier for me as a person running an event to get a camera to provide another view of the field than it is to get another referee to get another view of the field. And it’s kind of annoying that I can’t make use of the resources I do have to help with the problems caused by difficulty in finding resources in the form of better trained referees.
I would say that there’s two key differences. Firstly, and I must stress this, I believe there is immense value in having live referees watching the match from multiple angles. No question. Some violations you can only see in person. But some things will get past the live ref, and they might get caught on camera. (Or maybe they get missed by both, nothing is perfect).
Also, you earlier said
Just the viewing angles of a lot of the videos made this difficult. Parts of the field weren’t visible. Drivers would get in the way. It was hard to discern how many objects were in a goal (even with CU’s relatively large objects).
So I was responding to that point that, in theory, the viewing angles would be less of an issue if it’s not the LRT format.
For events that already have a livestream, this would not increase the cost. I am not arguing that events should be required to have a livestream. I am arguing that referees should be allowed to make use of resources available to them,
It’s my view that there’s good frustration when you lose because you simply weren’t up to the level required to win, and bad frustration when you lose because the situation was unfair. Good frustration makes you want to work harder and be more engaged to win, bad frustration makes you less engaged because “I did my best and then lost in QF because the ref thought I broke a rule when I didn’t”. And I want to do my best to make sure students stay engaged in the competition put on the by the Robotics Education & Competition Foundation
So, at least in our region we have a few very skilled refs who know the rules very well. However, most of our scorekeeper referees are less skilled. So, if you have one person, no matter how good, trying to watch 4 robots they will miss things.
The nice thing about a video replay is that very skilled ref can then go look at the video, knowing when and where a potential violation is about to take place, and watch the exact context of that violation.
Let’s have an example. The ref sees the red robot trying to get to the platform, and the blue robot is in the way. The match is over in 20 seconds, and the blue robot is touching the platform in the way of the red robot. Clear violation, DQ.
But then the ref looks at the replay. The blue robot was going to pick up a mobile goal. The red robot is playing defense to keep it from doing that. In the process of playing defense, the red robot pushes the blue robot into the platform. The blue robot is able to escape and the red robot follows, continuing to play defense on the blue robot, making no attempt to balance.
So with context we see that the blue robot should get the penalty for touching the platform, but not the DQ, because there was no attempt being made by red to interact with the platform so the infraction was not gameplay affecting.
This would be quite easy to see on a video looking back not in the heat of the moment, and almost impossible to see in real time, because there’s two other robots to watch. And you wouldn’t even need a great camera to see it.
A camera with poor video quality and angle can still capture a rule violation that a ref missed:
The blue robot can clearly be seen bumping the red alliance platform from this angle. This took place in the last 30 seconds, but the refs didn’t see it. A video replay can quickly and easily show that an SG3 violation occurred. That 30 point difference stopped us from moving on in the elimination rounds.
The issue I see is complexity. I volunteered once to help a MS tournament, not only did Tournament manager break down and their districts blend of tech restrictions cause all sorts of problems but many teams feel(sometimes justly sometimes unjustly) that only their interpretation of the rules is correct. This takes time, leads to a he said she said and generally wears out event staff pretty fast.
So lets say video replay is a thing, not only would it be entirely unfeasible for this event as it cannot get the simpler tech to run people would then question why video replays are allowed. Then we head back to Jon_Jack’s point
Is this accomplishing the goal of more stem?
in this use scenario no it wouldn’t be, it would be more like mainstream sports where half the people are throwing their hands in the air complaining rather than
Yo thats a cool bot
yea we should look into that mechanism
or the classic How in the world is this working?
just my two cents, please excuse my poor punctuation
Here in Wisconsin, most of our events are live-streamed, particularly if you rent the trailer from our non-profit Fox Valley Competitive Robotics’ Inc. However, a fair number of events are not. I have issues with Video Replay period and I don’t want to get into ALL of that. However, my concern is with the idea of making it optional for those events that can provide it. Those teams that feel strongly about having VR would/could stay away from those events that can’t do VR, And could end up missing out on an otherwise very well run event and possibly killing off the events that can not offer it. If VR is ever approved, it should be when ALL events are capable of doing it, not when only some are capable of doing it.
I’ve seen this point a few times. In our region, any event fills up pretty much instantly, and gets a waitlist on the order of like 20 teams. You could require teams to sign away their first born child and I’m pretty sure the event would still fill up. Is this not the case in other regions?
I think there are strong opinions on all front. Probably the best course forward is to do a field tests with early season scrimmages. Document the results and make a report of the findings. The report should include gear needed, training of volunteers of new system all volunteers, and recorded footage of key areas, interviews with teams, volunteers, spectators, and record of event pacing.
I think that would go a long way to forward a proposal to GDC about such a change. Right now not enough data, either quantitative or qualitative to make a decision accurately.
(a good test would be to run two divisions - all things equal (as far as training of volunteers about game) have one be Video Replay and one not.
[this could be a good post season experiment too using mature robots and teams]
70 team waitlists here. events are so in demand, I could probably set one up with a single field in my uneven backyard and get a full list of teams wanting to go. While I support events having the option of video replay, I don’t think it should be mandatory. We need events to be easier to run so there are more of them. But I see nothing wrong with an event being allowed to run video replays if they want to, that way the responsibility is on them to ensure that it’s fair and it doesn’t hold up the event, not on recf, competitors, or volunteers that don’t want to run it.
I keep telling myself to let this thread go, but I just can’t seem to resist the urge to continue posting…last post from me on this…maybe
There is some truth here. However, and I think @Grant_Cox mentioned this last night when he addressed this question: “As a coach, I tell my kids to ‘win anyway’”.
Teams are not going to be able to control what the refs see. Teams, to their credit, are trying to influence things here by asking for improvements to things outside their control. But at a given event, you can only really control your robot and your responses.
Refs will miss calls. VR systems will fail, or not capture the right angle. Teams will get unlucky schedules. Other teams will get luck schedules.
Champions “win anyway”. They overcome bad calls. They capitalize on others’ mistakes. They find a way to win. If you’re relying on a ticky-tack penalty swinging a game for you, you may not be trying to “win anyway”.
Every team loses. How do teams respond to those losses? Some teams blame others - the partner was bad, the referee was blind, they whitescreened. Champions look for internal reasons - how do I win a 2v1? How do I win so convincingly, that a ref isn’t needed to call the game? How do I ensure a good ranking even if I encounter unlucky situations.
Can VR right some wrong calls? Yes. Will it fix all? No. Are there costs (not just financial)? Yes. When VR fails, will teams still grouse? If teams go to tournaments with VR and then to ones without, will they grouse?
I go back to my problem statement that I think VR is trying to address:
As a competitor, I am concerned that the referees may not always see rules violations which leads me to believe that match outcomes are not always reflective of team skill
But there are other stakeholders involved in any solution to this competitor-drive user story:
As a referee, I don’t always see things. How can I best ensure rules are applied fairly?
As an EP, how can I hold a well-run event, that minimizes costs for teams, volunteers, etc.
As the RECF, how can I ensure a consistent event experience across a wide range of local conditions
No they don’t. That’s the entire point being made in this thead. No amount of skill or effort can overcome a systematic problem with the competition format.
i don’t believe this is a systemic problem
Big problem, maybe
Systemic i feel like is a stretch and would not be solved by a video replay it would be solved by better trained or more refs at competition
To champions point, @Mentor_355U is right, they might not get tourney champs necessarily but there is still excellence award and other strategies that give them opportunities to win
On the other point you raise, reasonable people can reasonably disagree about whether any amount of skill or effort can overcome a systematic problem with the competition format. I would posit that statement’s logical outcome is that winners and losers are determined randomly. I would argue that the fact that teams like Freedom Gladiators and PiBotics this year consistently win or make the finals of multiple highly competitive Signature Events is evidence to the contrary - that skill and effort are, in fact, the primary factors in determining who wins and loses.
