I have been a judge for 18 events since February 2019 (15 VRC, 1 VEX U, and 1 VEX IQ), including 13 where I judged notebooks and about 15-17 where I did interviews.
Here is some of my advice for new (or experienced) judges and judge advisors, based on what has been most important for me to remember:
- Read the Guide to Judging, and make sure you know it as well as possible. The Guide to Judging contains most if not all of the guidelines you need to know when making your decisions, and since everyone follows the same Guide to Judging, following it makes your events consistent with other events. Also take the Judge Certification course on the VEX website, which may include even more helpful advice. (Make sure you’re using the latest version of the Judge Guide, too, since it often gets updated to make it clearer. The last update was December 15, so if you haven’t read the latest version since then, I’d recommend reading the summary of the updates, and then the sections that were updated.)
- Before you finalize awards or send the judges or notebooks home for the day, make sure you’ve compared each award winner to the award description, and to any objective criteria that are relevant to the award (I will list some of these later). Also make sure you keep the Engineering Notebooks until the end of judging deliberations (as page 23 of the Guide to Judging says to), in case any of them might be useful for deciding between two teams for an award.
- Make absolutely sure every team gets an interview, and also make sure every team gets the same amount of time on their initial interview. Not every team needs to have their notebook scored (only the notebooks that qualify as Fully Developed), and notebook scores will never matter to a team except when they help them win the Excellence, Design, or Innovate Award. Interviews benefit every team, no matter how experienced they are, and no matter how much chance they have of winning an award.
- Follow the official order of precedence for awards: Excellence, Design, Innovate, Think, Amaze, Build, Create, Judges, Inspire, Energy, and Sportsmanship. For the ones before Judges, this is the order in which they are most likely to qualify the winner for the Regional Championship or for Worlds (Judges and the ones after it do not qualify for other tournaments). If a team is the top candidate for an award higher on the list, they should receive that award, rather than one of the awards lower on the list. Most importantly, whether a team wins Excellence and/or Design at a Regional Championship can affect whether they’re allowed to compete for these awards at Worlds (and if they can’t compete for Design, they couldn’t compete for Excellence, either).
- Make sure all teams who receive awards are student-centered, and they don’t have adult involvement that exceeds VEX/RECF’s Student-Centeredness guidelines. Also make sure that no one listens in on your interviews, especially coaches.
- Make sure no team at the event gets more than one judged award. (This has been followed at all the events I’ve been to, and the vast majority of events I’ve heard of, though on rare occasions I still hear about an event that didn’t follow it perfectly.) Awards not decided on by the judges, like Tournament Champion and Robot Skills Champion, should not affect whether a team gets a judged award, and neither should what awards a team has received at other events.
A few of my basic guidelines for awards:
Excellence: Best combination of notebook+interview, Skills ranking, and qualifying match ranking. I would usually value notebook+interview the most, and Skills more than qualifying matches. Regardless, the winner should be at or near the top of notebook rankings, as well as top 10 or top 30% in final qualifying rankings (whichever number is greater), and top 5 or top 10% in final Skills rankings (whichever number is greater), as well as being a top contender for other awards. This is the most important award, and if an event has spots that qualify for a Regional Championship or Worlds, this is one of them.
Design: Best notebook and interview who isn’t the Excellence Award winner. This is the highest-priority judged award besides Excellence, and qualifies for higher-level tournaments if the event has 4 or more qualifying spots. Other than Excellence, this is the most important judged award for a team to receive, especially at events where it qualifies for the next level.
Innovate: This is almost the same criteria as the Design Award, and I would imagine in most cases that the Innovate Award winner will be whoever was the second choice for the Design Award. The main difference I see is that the Design Award criteria mentions the team’s design process being iterative, while the Innovate Award does not. (The Design Award also mentions the team being student-centered, but this is necessary for all awards.)
Think: Based on having well-written and consistently-working programming, especially in autonomous mode, and doing a good job of documenting it in your notebook and describing it in your interview. The notebook is important to look at for this award to see what the program looks like, though the overall notebook score isn’t relevant to this award. If a team didn’t do well in Programming Skills, I would have second thoughts about giving them the Think Award, though I could perhaps still justify them winning it if their programming is extremely well-documented and they just had bad luck in Skills. If they don’t do Programming Skills at all, they probably shouldn’t get the Think Award, unless maybe they have a good program and just didn’t have time to make a Programming Skills attempt (though I’m still not sure if they can get it without a Programming Skills score).
Amaze: Consistently high-performing robot, and the students can explain how they worked together to develop it. Skills scores should be high, and high match scores are also helpful, especially if this team scores higher than their teammates do without them. (If allowed, I would use the VEX Via app to look at OPR rankings, and mention those teams in discussions for Amaze.) If one of the top teams in Skills had a good interview but isn’t scheduled to win Excellence, Design, or Innovate (or if I don’t know their interview score), I would bring them up in discussions for Amaze, and see what the other judges think.
Build: Robot is durable, reliable, and designed with attention to safety and detail. If a robot keeps falling apart during matches, or presents major safety issues, it probably shouldn’t win the Build Award, unless these problems were caused by factors outside the team’s control.
Create: Robot has a creative engineering design solution, and the team can explain how they worked together to design it and the rest of the robot. I’m not sure if the design needs to work well, but it does need to be creative, and working well is definitely a plus.
Judges: Can cover anything that isn’t already covered by another award but is deserving of special recognition. This can include factors not covered by other awards (such as exemplary effort or perseverance), or factors that were covered by other awards but where the team did not win those awards (such as a team that was a top contender for Excellence but lost to someone else for each of the other awards at an event). This award is required to be given out at all events, so at events with only three judged awards, the Judges Award could end up going to the 2nd- or 3rd-place Excellence contender. I’m not sure whether or not Judges is higher priority than Energy, Inspire, and Sportsmanship; Judges is listed higher in the order of precedence, but also says it’s for “attributes that may not fit in other award categories”, and three of those other awards are listed after Judges.
Inspire: Team with passion for the competition and positivity at the event. They might also overcome an obstacle or challenge and achieve a goal or special accomplishment at an event.
Energy: Team has a high level of enthusiastic and excitement throughout the event, and exhibits a passion for the robotics competition that enriches the event experience for all.
Sportsmanship: Team with a “high degree of good sportsmanship, helpfulness, respect, and a positive attitude both on and off the competition field.” The winner is “courteous, helpful, and respectful to everyone, on and off the field”.
Custom Awards: Events are also allowed to give out custom awards which are not listed above. These are lower priority than all of the awards listed above.
Event Partners can also give out Individual Recognition Awards, such as Volunteer/Mentor/Teacher/Partner of the Year; these are not chosen by the Judges, but rather by a process determined by the Event Partner (such as deciding on it themselves). I would recommend that Event Partners do not give this award out in a way that makes it look like the people choosing the winner might have been biased toward themselves or someone they are unusually close to (such as having the award go to the EP or to the person handing out the award without clear evidence that the winner had nothing to do with the selection of the award winner).
If anyone has other advice (or has a correction to make), let me know in the replies.