I’m new to volunteering with Vex this year and had a question about the Design award criteria. I’ve judged at multiple competitions so far this year and each one has handled this issue a different way. I want to try to come up with a way to handle it in the future.
Does the design award also depend on if a robot performs well? For example, if you have a notebook that is better than all the other teams’ but the corresponding robot has only won a single match, can they still win design? And if not, what is the cut off where we say a robot isn’t performing well enough?
There’s been arguments made both ways and the main one I hear is part of design is solving the problem posed by the current game. So if their notebook is fantastic, but they aren’t making any real improvements to their robot, but still documenting changes they are making, are they really excelling at design?
I just want to make sure that I’m doing best by the students I’m judging at the competitions and would appreciate any input others could offer.
I recommend you read this
https://www.roboticseducation.org/documents/2019/08/judge-guide.pdf/
and then take the Judge Advisor certification:
it is great you are volunteering so much and want to do better! We need more dedicated volunteers.
Judging process is separate from performance awards at event. You can definitely consider:
- Team interview information
- Engineering notebook
- Robot/team performance observed
Usually if all three connect, you have a strong candidate for Excellence!
Do not make the mistake of not awarding a judged award because you know they will get Skills and advance, the tournament format accounts for that.
Best!
I am a student so if you don’t trust me, then don’t, and I won’t mind at all. It would probably be imprudent to do so after all.
The fact that tournaments are taking robot performance into account for the design awards shocks me however. That is why I am making this post despite my own relative insignificance. I know what I am talking about, however, because my dad is an experienced judge at some of the region’s big events and my coach is one of the most experienced you’ll find in regards to judging and notebooks.
The design award is solely based on interview and notebook. These two things are a core of the RECF philosophy. Not only are they the criteria for the design award, the 4th highest award after excellence and the two TCs that is given out at a VRC or VIQC tournament, but they are also are the foundation of the excellence award (I believe this is stated word for word in the judge’s guide). A common misconception is that the excellence award is mainly based on robot performance. But no.
There you have a point. I think the interview will tell the truth as to whether they are really excelling at design. But anything at all related to the robot is excellence award only.
Just to specifically answer this question – no, the design award at all events should be determined based only on the criteria listed in the Judge Guide linked above, which do not include robot performance on the field.
I just had a side question, cause I was reviewing the Judged award overview on the Vex Judges Certification. The following is stated under the design award:
Judging the Design Award:
The Design Award criteria is the foundation for the Excellence Award and helps rank teams for other awards. Therefore, the Design Award judging process is fundamental to the judged award rankings and deliberations.
The Design Award is presented to a team that demonstrates an organized and professional approach to the design process, project and time management, and team organization. The team’s Engineering Notebook and Judges interview will demonstrate the team’s ability to produce a quality robot with minimal adult assistance.
The last line talks about producing a “quality robot”. So my question then becomes what is a quality robot? Because this training reads as saying that that’s a requirement to be presented the design award.
That is up to the judges, and is often complained about by competitors. Most judges will judge it off of how efficient it is, how consistent it is, and how it impacts the rest of the robot (aka very heavy mechanisms make take off points for this award).
I agree that is a bit confusing – my response would be two things:
-
What you’re judging for the design award is how well a team followed the engineering design process, and how well they documented & communicated that process in the engineering notebook and team interview.
-
The description says “ability to produce a quality robot” which is subtly different from “having a quality robot”. Note that the key criteria (listed just below the passage you quoted above in the judges guide) are all about different things the team “demonstrates” or “explains” in their notebook and/or interview, not about the specific features or capabilities of the robot they have at the particular event where you’re judging them.
If a team did a great job following, documenting, and communicating the engineering design process, then they should have the “ability to produce a quality robot”. This is not the same thing as performing at a high level in the qualification and skills challenge rankings at a particular tournament – obviously there should be some correlation between the two, but lots of other factors also influence where a team ends up in the rankings at a particular event.
(disclaimer: I have judging experience but volunteer more often in other roles. Other people who judge more than I do may have different or additional insights.)
This is my opinion which is a mix of what the RECF’s judging guidelines go by and my personal opinion
-
The Design Award is well, the Design Award. Whoever you think has the best design should win it. Although you should not base your selection off of performance ( since it is not a performance award ) this doesnt mean just because a team has a amazing notebook on a clawbot ( example ) with an outstanding interview doesnt mean they should get the award. It should be somewhat based upon the actual mechanics behind the robot and what it can do, thus the DESIGN award.
-
Deciding who should recieve the award could be in conjunction with the Judges award if being offered, if a team shows a good design process with a good interview, but their current design is not up to standards with other robots, then they would be a good contender for the Judges Award.
From personal experience I can say that many judges at reigonal competitions in NY seem to " Share the wealth ". For example a team with a slightly modified clawbot recives the design award which honestly doesnt seem possbile with the other teams that could have won it.
Hope this helps
As another team from New York (hey 7157x, I didn’t know you had a forum account) this is generally true. Judged awards other than excellence often go to teams that might not have performed as well but had unique designs or quality notebooks. We got design and build awards for Tower Takeover when we had a dr4b tray that didn’t do great in competition but we had a decent notebook and an interesting design.
Design is almost like a 2nd to excellence. Of course it has its own criteria, but if the excellence team “suddenly disappeared from the tournament”, the team running for design would likely get bumped up into excellence. To earn design, you can’t just have a unique build, you also need to perform decently. Then judges award mildly disregards performance, and finds something special about x team and why they should be recognized for something they did, still linked to team ability. Then energy, build, those awards are more fluff for the tournament, I’m not diminishing the teams that have earned these, because they still had to do something special to earn these above the rest, but they’re just a little smaller next to those big three.
Ditto for @holbrook - well stated!
This… is not true. Design Award is not a performance award. It is also not for “a unique build”.
Design Award is the foundation for the Excellence Award - you must rank highly in DA in order to be considered for EA. However, you can qualify completely for the DA and not meet the requirements for EA. What often happens (in my experience) is that a handful of teams will qualify for DA but only a few of those will then go on to also qualify for EA.
If there were no EA, then DA would still be decided between all DA qualifying teams. It may turn out that the team that otherwise would have won EA wins DA, but it wouldn’t be an automatic thing. It would, perhaps, be more a testament to the fact that a team who meets all of the qualifications for EA typically has done an excellent job of “demonstrating effective management of time, talent, and resources” and very likely can clearly show a good grasp of game strategy and highly effective communication skills & teamwork (all things that a Design Award winner is required to have).
It depends. Keep in mind that the foundation of the Design Award is the Eng Notebook - without the notebook, you aren’t even in the running for this award. I have MANY times seen a fantastic team with an exceptional robot have a very low-scoring notebook (or no notebook!) and not even make it into the stack for DA interviews. So, theoretically it is possible for a low-level type robot (clawbot. for instance) to have done a significantly better job of meeting the notebook criteria than a high-level robot.
Then you move on to interviews. Again, I have seen a clawbot team have a great interview while a team that you might think should place highly has a terrible interview.
Then the judge discussion. It is here that I would think most clawbots will / should (IMO) fall to the side (if they have made it so far) - assuming that there are other contenders with more elaborate designs that have also qualified. It is extremely likely that a more involved (and personally designed, as opposed to using pre-done plans) robot will be able to show a much deeper understanding of and repetition of the design process. I have never, at a high-school level competition I’ve judged at, seen a clawbot type get to the DA finalists. I have seen this at a middle-school level, though, and it has typically been largely based on the very first step (where the notebook is evaluated).
In simplest terms the Design Award is about the design process.
There are plenty of high performing robots with terrible dedication to the design process. There are low performing robots with students documenting and learning everything to the best of their abilities. At many events you’ll find the best process happens to come from a higher performing team because of their experience, but that is not always the case.
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.