How Do We Ensure Consistency Across Signature Events? (Google 2020)

The Google Signature event highlighted a number of concerns with Signature Events. VEX and/or the RECF must have a larger influence on the planning and execution of these events. Overall, according to some participants, the event was run well. However, it had some serious issues, at least for those looking from the outside, in.

Event Requirements


Here is the signature event requirement document that will be used for the next part of this post.

TR.1. There shall be no more than 50% of registered teams from the same region

Out of the 56 teams in attendance, 36 were from California, according to Robot Events, though this may be due to California being split into several constituent regions.

TR.4. Teams shall be given the opportunity to run three (3) Driving Skills Matches and three (3) Programming Skills Matches

On the first day, they allowed unlimited skills matches, which was talked about in detail here: Why is the Google Signature event allowing more than 3 skills runs?

TR.5. All competition fields for Qualifying Matches and Finals Matches shall be raised off of the floor to the same height and be between ten (10) and twenty-four (24) inches.

The fields at the event were not raised.

TR.6.4. There shall be at least (1) Practice field for every 16 teams.

According to those at the event, there were 2 practice fields. There should have been 3-4.

L2.3. The event shall be a two (2) day event, with at least three (3) Qualification Matches on each day.

The event was 2 days (for HS), however, the first day only consisted of skills matches. No qualification matches were played.

L2.4. The event shall broadcast to VEX Via live during the event.

While the event was broadcast live at some points throughout the day, it was spotty, and it occasionally took hours before scores were posted.


This begs the question- What does the RECF do if an event doesn’t meet these requirements? What is the point of having a document outlining what is required out of a signature event, if it isn’t enforced?


Reffing


It is hard to write this without sounding depreciate of all that referees do. We mean no disrespect to any of them.

The reffing at the event was less than ideal. On two occasions the referees said they did not see violations, and had questionable interpretations of rules.

For example, one thing that was said during the drivers meeting (according to teams at the event), and one thing that teams were warned for, was descoring stacks made during autonomous, in an attempt to put a higher one up. The rule they were citing was SG6.

Keep Cubes in the field. Teams may not intentionally remove Cubes from the field. While Cubes may accidentally leave the field when attempting to Score, doing so intentionally or repeatedly would be a violation of this rule. Cubes that leave the field during Match play, intentionally or unintentionally, will not be returned.

The first seed had to switch to a lower scoring autonomous, so the cubes could be descored without the potential for leaving the field at all. The presumed intent of this rule is to prevent the making of a strategy which is to intentionally, and strategically, remove cubes from the field. There was a clear attempt to keep these cubes in the field, and it gave them an advantage during the match to do so. A few examples of this in play were at 9:06:26, 9:39:42 and 9:56:20.

Changing their autonomous made them lose 3 points towards the autonomous bonus each match, changing their entire strategy.

Here is the link to the stream.


There was another post made on the forum here about the reffing. I disagree with the manner they went about explaining their side, so I won’t link it. Here is the video they provided.

This is an obvious and clear violation of SG3 A, E & D by 7K. They indirectly contacted a robot in the IPZ, and the goal zone barrier. As laid out in Q&A 432, this is a clear disqualification. Note the middle referee pointing at 7K’s robot. The same pointing occurred in Finals Match 1, mentioned below, but neither violation was called.


Link to Finals 1

During Finals Match 1, at approximately 1:47, the blue alliance crossed into the inner protected zone volume. As laid out in Q&A 449, because the blue alliance was blocking the protected zone and was then pushed into it as a result of offensive maneuvers by the red alliance, this should be a disqualification for the blue alliance.

185A violation 2

During the same match, the blue alliance crossed into the inner protected zone volume at 1:19, which is also an automatic disqualification.

185A violation 1


It would have been potentially easier to spot violations of these rules if the referees had been at the unprotected zone corners, instead of all at the back, as said at 3:32 in the Chapter 2 Referee Training Video.


Conclusion


We aren’t trying to change the outcome of the event. We aren’t trying to disqualify the event or to disrespect the Event Partner(s). As new as Signature Events are, the RECF is surely still trying to figure out exactly what they are, and what they want from them. We want to use this as a talking point that there is still work to be done, and conversations to be had about these events.

It’s hard to say “what if” to if these issues would have changed the outcome of the tournament, but we can say that all of these issues could have changed it.

As for solutions for this problem, we believe a combination of RECF/VEX involvement in the organization and planning process, execution, ref training, and community input would go a long way. We hope this is the first step into improving these events, and making them something that we can all be proud of.


This post was written by myself, in collaboration with @LegoMindstormsmaniac, @jetc17, @javathunderman, @Sylvie, @Nolan_XD, and Bryce from 12T.

67 Likes

Interesting points, and definitely some things that the RECF should be monitoring. I’m a bit surprised to hear that a major event wasn’t following the requirements as listed in the document you referenced; I wonder if that was something they set up authorized changes to in advance?

Perhaps the RECF needs a post-event survey sent to all attending teams where they get more specific feedback that they can then use to address issues in future events. That might actually help some of the issues that small, local events have, too (where there are some definite mis-understandings of certain things that could easily be addressed & fixed for the future if brought to the EP’s attention by someone in authority, like the RSM).

I will address this point, however, as I’m a little confused as to what you are pointing out. IMO, this is a correct interpretation of the rule regarding keeping cubes in the field. There was an official Q&A regarding this (https://www.robotevents.com/VRC/2019-2020/QA/462) that would lead me to the same conclusion and I think it was reasonable of the refs to address this in the driver meeting and enforce it in the event. Teams should be aware of this as a rule and should plan their autons accordingly; either have a smaller stack or develop a way to “most often” be capable of descoring the tower without having the cubes tend to leave the field.

The best way to show clear intent to keep those in the field that I can think of is to keep them in the field… If I were to head ref that, I would require that a team be able to show that at least the vast majority of the time they are able to remove auton cubes without losing any. That clip at 9:06:26, for instance, does not show any attempt at all to keep those in the field as far as I can tell. I can’t see them ever keeping the cubes in the field with that method, which is exactly why it is an SG6 violation. The red robot at 9:39:42 does an excellent job of working to keep them in the field and does so successfully - I don’t see how a ref could warn against that one (as they are doing exactly what all should strive to do); the BLUE robot, however, literally just crashes into them. In the 9:56:20 clip, they aren’t doing as well keeping them all in play (probably because it’s a slightly higher stack) and should be cautious.

(Keeping in mind that it’s a little hard to know what robots are doing what things in those clips and I have no real idea which actions warnings were actually called on and which did not get warnings, so I may be misunderstanding what you are pointing out…)

Also, I’ve only really watched thoroughly the finals matches with the first seed but it looks to me like they consistently have a 7 cube auton that they don’t descore in those matches. Not having seen many of the other matches or knowing anything about what their strategy for the game was / wasn’t, it appears that they found a way around the issue (or were you saying it was their partner’s stacks they had to adjust?).

6 Likes

From my current understanding, they were warned after Semifinals, which is the match they made a clear attempt to keep the cubes in the field, and one ended up leaving. It should also be noted, I am talking about Red Alliances (7K and 2114X) in this context. They are the ones who got the warning, that I know of.

Sometimes this isn’t a real possibility. Personally, for our robot, it is best to ram the cubes, as the top cubes generally fall in the opposite direction that the bottom cube is pushed, at those speeds. It really is a gamble as to how many, if any, will fall out. It still doesn’t meet, in my opinion, the definition of intentional.

7 Likes

I wonder if there was a RSM there or any RECF oversight. I know most signatures events there is a RSM to make sure all these guidelines and rules are being followed.

SG6> Keep Cubes in the field. Teams may not intentionally remove Cubes from the field. While Cubes may accidentally leave the field when attempting to Score, doing so intentionally or repeatedly would be a violation of this rule.

(bolded by me for emphasis)

I think it is the “repeatedly”, here, that often causes the SG6 violation, and the fact that if repeatedly you tend to knock them out it becomes pretty intentional (if you continue the action, knowing they are likely to go out). The limits for this are at the Head Ref’s discretion, according to the GDC Q&A.

A large percentage of the time, when I see robots ram stacks that are over a few cubes high at least 1 cube (usually more) fall out of the field. Therefore, if that is the method a robot is using, if a ref I would tend to say it violates SG6 (if done repeatedly). I guess teams who do it like that will just be taking a gamble at any event they go to as to the limit at which the refs involved will enforce this.

[ETA - if 7K did as good a job in the majority of their matches as they did in the middle one listed above keeping the cubes in play, I wouldn’t personally have warned them for 1 cube once or twice, as that wouldn’t rise to “repeatedly” for me; however, that would totally be dependent on the activity throughout the day…]

Anyhow, back on official topic. :slight_smile:

For the inner protected zone violations, it doesn’t look like that was enforced correctly if they weren’t calling them (you did a good job pointing those out in the pics, btw). That can be very hard to see in fast game play, but with multiple refs and it being a DQ offense you’d hope it would get called more.

I agree with @ZackJo - I wonder if an RSM/RECF member was there and what discussions might have taken place. Transparency with that sort of thing can be really helpful in this day & age, where us “armchair warriors” can watch videos and pick out particulars from far away… lol.

12 Likes

About the zone, iirc, the rules state that it is if you contact the ipz, not if you break the plane. Auto line is also tile contact. Full containment in the opz is plane based, though, from my interpretation. I think vex needs to clarify what is a violation by plane break and what is a violation by tile contact.

Because the IPZ is a 3D volume, not just an area on the foam tiles, “contacting the inner protected zone” and “breaking the plane of the inner protected zone” are equivalent. See Q&A398.

12 Likes

The original post makes some great points about the overall requirements for Signature Events. Instead of focusing on individual referee rulings, let’s bring the thread back to the intention - how to ensure consistency and adherence to the guidelines at these events.

1 Like

Regarding the 50% rule. We ran the Signature Event at Lambeau Field. We were given a date by RECF that if we had not acheived the 50% out of state/region teams, then we were allowed to fill the event from local teams. Otherwise, we met all of the requirements. Jim Crane from RECF was in attendance along with our RSM. We had 1 practice Field for every 14 teams, raised fields, rand qualifying matches over two days and was both livestreamed and was on VEX Via.

10 Likes

The REC Foundation agrees that Signature Events are still evolving and that there are still opportunities for improvements. Jim Crane is already tasked to evaluate the good and bad from this past season and make recommendations on changes for next year. We are committed to reviewing these proposed changes with the community for feedback.

This is a great idea! @JimCrane

Thanks for the constructive feedback.

  • Dan
26 Likes

Can we please address the “Judges Station” setup?

This was super inconvenient at the Google event and it seemed to miss a key component of the judging process: the observation of teams as they work. I truly enjoy it when judges can comment on what they NOTICED. Cooperation with other teams in the pits/ on the field. How a team moves from the match to the pits, especially after a loss. Do they make a plan on the walk back? Bicker with each other? Blame the refs?

Here’s how Google ran Judging:

  1. Teams were called to queue for matches very early, 7-10 matches ahead of their own.
  2. Teams were required to queue for Judging between matches. But, no time to wait in both lines so they were told to have one member as a placeholder in the Judging line… which made that member unavailable to even watch the match!
  3. So many teams treated Judging as a presentation instead of an interview. It was WEIRD, and yet that’s what it seemed the judges were expecting? Prepared speeches and display boards, like Science Fair?
  4. Over 50 teams in the event. A perfect time for judges to award some “other”/ non-advancement awards, but they elected to not do so. Seems like a missed opportunity to recognize additional teams. There were some great ones there.

I understand both the need for efficiency and the difficulty in finding volunteer judges. But please tell me RECF is not moving toward this format. It’s way better when they surprise a team in the pits for an INTERVIEW!

17 Likes

Well, that’s… just weird. And another area where it sure appears that the Sig Event didn’t follow the given Vex guidelines.

According to the judge’s guide:

Pg5 Judges should spend time observing teams in the competition area. Judges can validate statements made by teams during their interview and can evaluate their robot performance and game strategy. Additionally, Judges can assess a team’s sportsmanship, energy, and enthusiasm while observing them in the competition area.

Pg6 The pit area is the home-base for teams at the event. […] Judges at local events interview teams in the pit area. Judges at events that qualify teams directly to VEX Worlds and judges at VEX Worlds will interview teams in the pit area.

15 Likes

The signature events have one huge flaw, in my opinion.

Signature events rely on teams from multiple, and hopefully many, regions signing up to take part.
However, most team cannot afford to fly out or travel long distance to attend these events.

The end result is that signature events are either:
a.) Mostly local events.
b.) Elite

6 Likes

I think they become mostly local unless they are early enough in the season. Nobòdy wants an international trip when there are state and national championships all round the world to be thinking about. But I don’t think they are elitist. They just become regional once you pass the international registration deadline.

2 Likes
  1. These events are way too expensive for any normal regional team
  2. Those elite teams go to signature events because they have money and want to practice with other elite teams before worlds
3 Likes

It is not much more expensive to attend a signature event in your region then a regular regional tournament.

Last time I checked the google signature was 200 (correct me if I’m wrong) and a regional was about 50 (in North Texas)

The Google signature event was 200 however it is typical for a regional in California to be about 150.

4 Likes

“Ram the cubes”

This is a violation. Full stop.

Talking about top/bottom cubes here makes me think they are already stacked in the corners, making them scored. This is completely disallowed in SG3. You would be disqualified or given a warning, depending if it is match affecting or not.

Think the context here is about descoring your own stack (scored during autonomous), not about descoring your opponents’ stacks (which is mostly DQ-able).

I really think a lot depends on how fine a line does GDC expect the referees to draw?
Is it intentional for the cubes to go out of the field? Definitely not.

But @TeamTX is right to mention about “repeatedly”.
Still… will be nice if there is an official stand about it and this rule can be implemented evenly throughout all events.

Edit: or… was there already an official Q&A on it? :stuck_out_tongue:

6 Likes