Replays/Video Review

21" Monitor with full zoom. Not janky cell phone stuff, real images.

3 Likes

For a video replay to work, especially a cell phone video, the students would need to be fully prepared before they start talking to the referee, and be willing to obey any restrictions placed on the process (even if they don’t agree with all of them).

Before they could start showing the referee the video, the team (and their coaches if possible) would need to (1) watch the video themselves, (2) pinpoint exactly what information in the video will prove their point, and (3) make sure the referees would be able to change their minds by looking only at the video, within the 2-3 minutes they might be allowed to, and preferably without needing to squint (if making the referee need to squint was even allowed).

When the team’s time is up, they need to thank the referees for their time, snd respectfully accept the referees’ answer, no matter what it is. They should be especially considerate if a referee needed to look closely at a tiny screen or make out something hard to see, or if there were other factors that made the review process harder or more stressful for a referee.

Even with these restrictions, though, I still see three major drawbacks to allowing cell phone videos:

  1. Some teams may be able to afford phones with more storage space for high-quality video, or with better cameras, than others can.

  2. Some teams may have more people available to take videos than others do, so they may have more chances to get an angle that shows what they want it to.

  3. No matter how many restrictions are put on video replay, there will always be people who violate those restrictions, whether on purpose or by accident, or who unintentionally wind up making things hard for the referee in some other way.

2 Likes

Lol I don’t even know how to respond to this. EPs are important, but it seems like every single post you make is “EPs matter, I’m an EP, so therefore you’re wrong.” You’re not getting any apology from me or from anyone else.

These are both fair points, I guess an event would need official recordings for a replay system to be fair. Maybe the RECF could beta test a replay system at an event that is already streamed and then spread the system if it’s successful?

3 Likes

I appreciate the points the OP makes… however…

When I initially got involved in Competitive Educational Robotics, as an FRC mentor 15 years ago, I thought it would be a great way for students to develop technical skills and encourage students to consider technical careers.

I wasn’t wrong… but I only saw a part of the picture. By far the best part of my involvement has been watching students who were already technically interested develop their teamwork and communication skills… AND experience the life lessons that one learns from playing sports.

One of the lessons that you learn in sports is that sometimes you lose because of a bad call. You can whine, you can complain… or you can suck it up and say, “I let it get too close… I should never have let it get close enough that a ref could decide the result.” It took me about five years to realize that after our basketball team got knocked out of provincials by a shot that was taken after the buzzer sounded. (Cough… no bias on my part… cough…)

Yeah, you can argue, “But it’s not RIGHT! It’s not FAIR!” until you’re blue in the face… I understand. We tried that. We still lost. We sucked it up and got on with life. Apparently life was still pretty good, despite this horrible injustice that was foisted upon us.

Part of the deal when you sign up for a competition that has judging and reffing is that you accept that the judges and refs will do their best, be honest, and that while they will sometimes be demonstratably WRONG… you’ll accept their honest error. Heck… sometimes you have to accept their dishonest errors… read up on officiating in judged sports at the Olympics.

So no… I don’t want video replay delaying tournaments. Sorry… but we’ve got teams that need to catch a ferry to get home. We’ve got volunteers who have commitments for the evening. We’ve got a clean-up/take-down crew who would like to be home in time to enjoy a bit of the evening. I’m perfectly okay with seeing a team knocked out because a ref made a mistake. Sorry, but while the teams are an important part of the tournament… they’re only one part of the event. We try to provide the best possible experience, but aren’t going to be setting up multiple cameras and a video review station with a direct link to Dan’s desk, nor do we want to wait for teams to pull up shaky, inconclusive cell phone video shot from the stands and argue their point.

In short… I understand. I’ve been there. I’ve felt the same way. I know why you want a review and if it were quick, simple and easy to do then I might be interested. But you can actually learn a lot from having a bad call go against you… because sooner or later you’re going to have a bad call go in your favour. It’s called sucking it up and getting on with life with a smile on your face. It won’t be the only time you get hit with a ‘bad call’… or get lucky when someone else gets hit with one.

Sorry… but no.

Jason

15 Likes

This is true; sports (including VEX) can teach many valuable life lessons. Learning to stand up for oneself can be included in those, along with knowing when to shut up.

…or you can respectfully make your case to the ref, and, if he/she upholds the incorrect call, try to come up with solutions to better enable refs to make the right call. This is engineering; you find a problem, come up with a solution, and implement it. You try to make things better. Throughout history if marginalized groups in society had just said “Well, I guess being treated like a sub-human is how life works, so lets just suck it up and move on with a fake smile”, then things would have never gotten better. It took individuals and groups pushing for change, for equality, for solutions. Obviously, this particular issue pales in comparison to that of civil rights, but it is the mindset I have particular issue with, not the particular application.

I’m sorry, but I find this sentiment disturbing. I’m not “perfectly okay” seeing an injustice (however minor) occur. If it is within my power to fix or otherwise do something about, I would do so.

Maybe there’s a disconnect here, but aren’t the teams (and the students that are part of them) the entire point of the tournament? Yes they are indeed only one part, but they are the part around all other parts revolve. This is not to say that all the other parts (EPs, refs, other volunteers) must bend to the will of the students, but rather, what EP, ref, judge, queuer, or inspector would volunteer their time if the students were not present? On the other hand, students would still build some robots (albeit to a much lesser extent) in the absence of the other parts of a competition. I don’t mean to come off as an entitled brat (please gosh no), but multiple EPs on the forum here seem to have missed the fact that many of the teams have spent just as much time or more working throughout the year, with much sacrifice of their own.

Considering the hundreds of hours of time devoted by both sides, neither should be put out in the cold for the other’s benefit.

This is a strawman. No one is asking for this kind of ridiculous action. Rather, what has been proposed here (by several users, not just by the OP) is much more achievable by many events. It may not be possible for all right now, but it is a step in the right direction. Personally I think a step in the right direction is better than acceptance of the flawed status quo.

Is this supposed to balance things out? Why would I be happy knowing that I “won” because another team was treated unfairly? The stupid trophy itself is worth nothing to me if I didn’t earn it. If something like that happened to an opponent of mine, then I would still be very much in favor of some solution to the problem. In fact, this has happened to an opponent of mine, and it is very possible that such a video system would have overturned the call and I would be able to rest easier knowing that justice was done, regardless of whether I actually won or lost.

One more thing:

Please understand that many (not all) of these bad calls happen in matches that are far from close, or a robot was illegally tipped. In these cases the teams have done all that was in their power, yet the ref overturned all of it.

Aside from that, private universal casting of blame upon oneself is not a healthy thing to do, and is not a good “life lesson”. Sometimes other people make mistakes, and there may truly be nothing that you could have done to change what happened; so do what you can to improve and learn for the next time, and don’t blame yourself for what you can’t do. That is the life lesson here.

4 Likes

Foster, please understand that not all 3100 members of the discord act in the same way or believe the same things as any individual member. It just isn’t fair to group people together like that, just as it wouldn’t be fair for me to say “The VEX Forum needs to pull it together”, or “The VEX Coaches Facebook Group needs to pull it together” because of the actions of an individual or individuals from those groups.

And as far as your repeated condescending mentions of the discord, please stop. It is very frustrating to be repeatedly antagonized by a user who does not even see first hand what happens within it (but I would love to have you join, I think there is much respectful discussion that could be had if you joined). As far as you being mentioned on the discord, I would not be surprised if my username came up on the facebook group or other EP group if I made it a habit of talking in a similar manner towards one of those groups.

17 Likes

I have no particular expertise to comment on the technical issues that would have to be overcome in order to allow video replays. So, I’m admitting up front that I don’t have any good suggestions for how to implement. However, I will say that in my own (painful) experience this year, a video replay being viewed would have averted a series of events that should not have occurred in my own area. No, I’m not going to go into detail at all on here. And yes, I used to be one of those who would have argued against video replays. But my own experiences of this year have changed my mind on the video replay question. I do think it is a valid question and should be warranted in certain parameters - perhaps semis and finals only. I also think it could save having to get upper RECF folks involved in investigations that eat up their time.

4 Likes

This American Life just put out a great episode about how the NBA now handles replays and video reviews. They spent double what the annual RECF income is to build their “replay” room.

I’ve heard from many in the RECF that their main concern is consistency across events. There is no way they can implement a replay system consistently. RECF is trying to certify referees to make sure they read the materials and show some kind of competency.

I also feel that they also need to try and design the game in such a way that reffing is a little easier. The expansion zone is hard to ref. Pinning is hard to ref. IQ events are inherently easier to ref as the interaction between the robots is always positive. VRC games of late are inviting a lot of interaction between the bots and make the game more exciting, but harder to manage. Perhaps they should try and swing the other way for the sake of ease of reffing.

12 Likes

Full disclosure - we have streaming at our events as well at least 3 GoPros (1 is extra but I can set it up for an additional view).

I’d be ok with doing some form of limited trial at one event I run this season, depending on game and the rules. We would also need to define when it’s appropriate to be employed, and what we would review, and what the possible outcomes of it are.

Maybe at Worlds or the EP summit this year, I’ll get the chance to talk to @DanMantz or other people within RECF to see how this could be possibly drawn out.

I can tell you coming from both sides, the hardest part is minimizing the effort it takes to implement this. Maybe this isn’t something your run of the mill event can’t do, but with Certified Referees at a Signature Event, another World qualifier, the US Open, it might be doable, but still comes at a cost.

11 Likes

@Foster has made this suggestion several times but I think it’s getting overlooked because of all the mud-slinging going on here.

Those of you who want replay so badly - why not give it a shot? Find an event, contact the EP, and work with RECF to see if they will approve a pilot program. Produce a document describing how it will be done. Get RECF, EP, and referee signoff on your documentation and distribute it to the teams. Find your volunteers to set up the cameras, recording, etc. and see how it goes. Document your results, share with others, conduct postmortems and see if it would be viable to roll it out further.

19 Likes

Yep this right here.

I’m down for doing it as a pilot. I think I can try and implement some small limited form of it and see how it could work.

4 Likes

Wrong place, but yes

1 Like

I do not know if this has been said already but there was actually 2 miscounts on both red and blue side for auto which would have still ended in a tie

2 Likes

To start everything off, VEX Worlds is more than capable of having replays during their event. There are at minimum of 2 cameras that always run when a field is in-play. It would be really nice to see if VEX can be able to run replays. A good example of replays being used in a livestream can be at VEX Worlds Round Robin, as well as I have seen some occasional video replays on a VEX British Columbia livestream. In terms of normal events, it will be nice for events to give the option to offer video replays, but I don’t believe it should be a requirement as things would get complicated with some events that barely even can afford being ran. in terms of your topic of having teams be able to record matches on your phone, imagine how distracting that will be. Imagine a competition that turns from robotics into “see how much you can record every match with your phone so we will make certain that we don’t lose autonomous.” If every person is having to record their matches with their own phones then there will never be social interaction, bonding, and teamwork. You are replacing ethos and the environment to try to have a perfect scoring system. Perfection will never go in line with feelings and social interaction. Perfection is boring and is never possible in a robotics competition that its sole purpose is to provide a competitive, collaborative, and bonding environment. Now, as I said though, if events have the ability to livestream and they do livestream, they should also have the option to accept requests for replays but the requirement must be that the team must be following G1 not just repeatedly but thoroughly. You will be hitting two birds with one stone in these cases by encouraging further respect to referees as there is something to gain, a special privilege, if teams are respectful and not just something to lose if they aren’t. As young people are, breaking rules is inevitable as people like rebelling a lot of times. If referees make video replays be a privledge from respect, then there will be something to gain. Replays are relatively easy to do in a livestream, as they can be initiated with a simple button press or just by going on your phone and actually viewing the timestamp of the stream, showing it to the referees.

@DRow

5 Likes

Worlds would be a bad choice to try to implement this for the first time. There’s several reasons:

  • Worlds matches have very fast turnaround time, some of the fastest of the season.
  • Worlds has a very tight schedule to try to fit in as many qualification matches as possible. A few minutes delay here and there to review replays will really add up.
  • You’d need to implement it across 10 different divisions - that’s a lot of volunteers to recruit and equipment to provide.
  • Divisions do not have any suitable equipment that could be used to review match footage. Doing so on the scorekeeping computer is not an option. You’d have to provide a second laptop or PC for each division.
  • Doing it only during Round Robin has many of these same problems - the matches happen fast, there’s a ton going on in the background (preparing for the finals), and the finals always run late anyway so there’s no appetite from anyone in charge for anything that could add any more delays.
  • Not to mention presumably you’re thinking 2019 Worlds and not 2020 - it’s only 2 weeks away. That’s not enough time to implement it all (including documentation and clear rules on how and when it would be used).
  • Probably most importantly - the people running Worlds are not (currently) in favor of allowing video replay (I’m taking an educated guess here based on the fact that they wrote the rules). They are not bought in to the idea, so why would they want to invest extra time and money and effort (when they’re already working crazy long hours every day to get Worlds ready anyway)? Again, if you all want it, then you should go make it happen. Find a willing EP and then you go do all the work and provide all the equipment for the test run. If you can prove that you can make it work then RECF might be more willing to consider it.

Seriously, go work with RECF to see if they’ll let you pilot it at an early 2019/20 event this summer or fall. You could even run a post-season Turning Point event this summer for fun that’s unofficial and then you could do whatever you wanted without even getting RECF approval.

10 Likes

On top of that, they contract out all the AV for Worlds to begin with. That’s an additional expense the RECF would have to provide which starts to get costly. I think Worlds is hard for them to even break even on.

1 Like

If it did happen at worlds, it won’t be the first time VEX/RECF will be experimenting at VEX Worlds Cough BO1 Cough
VEX/RECF, although are organizations to host events, create robots, etc. They are also built to experiment and try things out for the future generations. VEX and RECF love investing, which is what makes them so successful.

I will admit that they have one of the fastest turnaround time in the season, but I bet there can be some workaround to allow matches to run while a team is showing footage. I once had a 15-20 minute discussion with a referee as the matches were running, and there didn’t seem to be any problems.

The equipment is already there, and you aren’t changing the staff required.

There are multiple computers running, if I’m not mistaken. One is used to update the score and the other is to explicitly stream.
But, if it’s not possible to have the streaming computer also do replays, if another laptop is required for worlds, we must realize that VEX is using Ronin camera equipment worth tens of thousands of dollars, maybe even got into the hundred thousands. Every year VEX creates a budget to improve things, I bet they will have the funds to buy laptops if we give this a good year of implementation.

I don’t care about the year, as long as it’s implemented.

That’s a pretty good idea, I think I may look into that. The only issue is that I don’t believe I have the camera equipment (heck even the funds for it) to afford streaming multiple fields (I calculated that it will cost me around 250-300 to be able to have the equipment to do such). I will most likely need to borrow equipment from some other group or organization.

4 Likes

The footprint and level of effort of implementing a replay system vs a competitive change like BO1 is wildly different. Two separate things, you’re comparing apples and oranges.

I will admit that they have one of the fastest turnaround time in the season, but I bet there can be some workaround to allow matches to run while a team is showing footage. I once had a 15-20 minute discussion with a referee as the matches were running, and there didn’t seem to be any problems.

Not as much as you think. My team had a dispute with a ref last year and the ref had to go back to reffing matches to stay on schedule while they waited for the overall head ref to come and talk to them.

I forget what the cycle time is for Worlds but it’s something insane like 2 minutes and change.

The equipment is already there, and you aren’t changing the staff required.

You’d actually need additional equipment and staff. Like that staff already has a role and specific job. Reviewing match video for ruling issues requires additional staff/volunteers. It’s hard enough to get volunteers for Worlds. This makes it harder.

3 Likes

I think that video reviews should be added. My team attended the Create US Open tournament as well and lost a match after a clear entanglement was not called. This missed call caused our alliance to lose their first match of the day late in qualifying. We argued that the match should be redone but the referee stated that there was no intent to entangle our robot. Intent doesn’t matter when it comes to entanglement. If you entangle another robot, purposefully or not, you should be DQed. The eliminations had many missed calls including the ones you mentioned. Video replay should be added to help eliminate the mistakes made by referees. I am hoping that Vex will add this component to next years game.

1 Like

The purpose was to say that VEX is willing to take risks to improve the competitive environment. Effort is not the purpose or reasoning of my first couple of sentences.

I doubt equipment would be a big problem at worlds, and I don’t believe the impact on staff will be as bad as people would think. In fact, I think it will save time considering that instead of having a large discussion about rules and whether a team did said illegal action or did not do said illegal action, the referee can just simply rewind on the feed online and finish the discussion within minutes, maybe even seconds compared to a 1/2 hour discussion to attempt to clarify.

4 Likes