Let's talk about awards

VEX Robotics Competition Discussion VRC > Previous Games
1

For the last few days, there has been some discussion about how awards are given out at local competitions. That’s kind of off-topic from what the original question was, so I thought we would just go ahead and have an actual, rational discussion about concerns, judging criteria, or just general thoughts about how these awards work.

I’m just going to pull the full list and descriptions from the Appendix E of last year’s manual.

*World Qualifying Award at competitions of any size
*Top All Around Team (Robot Performance & Judged)
Tournament Champion Award
*World Qualifying Award if the competition has over 24 teams
*One trophy for each team on the winning alliance
Design Award
*World Qualifying Award if the competition has over 42 teams
1. Engineering Notebook is a clear, complete document of the team’s design process
2. Team is able to explain their design and strategy throughout the season
3. Team demonstrates personnel, time and resource management through the season
4. Teamwork and interview quality
Volunteer of the Year Award
*Recognized Event Volunteer
Sportsmanship Award
1. Team is willing to help others by sharing resources, knowledge, and encouragement
2. Team has helped not only alliance partners, but all teams, by sharing resources
3. Team has enriched local VRC events by volunteering personnel and/or resources
Judges Award
*Judges’ recognition

In addition to these standard awards, it is expected that many tournaments will offer a combination of the following awards:

Tournament Finalist Award
*World Qualifying Award if the competition has over 100 Teams
*One trophy for each team on the runner-up alliance
Amaze Award
1. Robot design is consistently high-scoring and competitive
2. Robot autonomous mode is consistently successful
3. Robot is robustly constructed to fulfill its designed task
4. Teamwork and interview quality
Build Award
1. Robot construction is of professional quality; robust, clean and elegant use of materials
2. Robot efficiently uses mechanical and electrical components
3. Robot is designed with detailed attention to the hazards and rigors of the competition
4. Teamwork and interview quality
Create Award
1. Robot is a well-crafted, unique design solution, demonstrating creative thinking
2. Team has demonstrated a highly creative design process and methodology
3. Team has committed to ambitious and creative approaches to playing the game
4.Teamwork and interview quality
Energy Award
1. Team maintains a high level of energy throughout the event
2. Team demonstrates support for all participants
3. Team’s positive energy enriches the event experience for others
Innovate Award
1. Robot design demonstrates an ingenious and innovative piece of engineering
2. Innovative feature is soundly crafted and is an effective solution to a design problem
3. Innovative solution is integrated as a part of an overall well crafted robot
4. Teamwork and interview quality
Programming Skills Champion Award
*Top Programming Skills Challenge Team
Robot Skills Champion Award
*Top Robot Skills Challenge Team
Think Award
1. Autonomous code is effective, cleanly written, well-defined, and elegantly designed
2. Team has explained a clear autonomous strategy as part of an overall strategy to win
3. Team’s autonomous code is consistent and reliable
4. Teamwork and interview quality

Source for the award descriptions
Source for the World Qualifying Criteria

Now, there’s more to the judging of several award (Excellence, Judges, etc.) detailed in the .pdfs at the bottom of the quote. I just didn’t feel like it belonged in the initial post.

So, I guess my questions for teams are these:

  1. Which (if any) of these awards do you feel is most often incorrectly awarded?
  2. How would you change the described judging criteria to encourage fairer distribution of awards?
  3. Do you feel that any of the awards that are not currently World Qualifying should be?
  4. Do you feel that any of the awards that are currently World Qualifying shouldn’t be?
  5. What aspects of the VEX Robotics Competition do not have an award dedicated to them that deserve one?
  6. What aspects of the VEX Robotics Competition are too heavily weighted in the number of awards they involve?
  7. What has been your experience with the Interview Process at local competitions?

I also have some questions for those who judge local competitions.

  1. How do you judge an interview?
  2. How do you judge Engineering Notebooks?
  3. What stands out to you about teams?
  4. Do you have any advise for students to take into mind this year?
  5. Do you use the rubrics provided by VEX or do the competitions you help at have their own?
2

I find that the Innovate Awards are almost always awarded in a fashion that I don’t understand. There’s nothing innovative about the “best” intake at a competition when it’s just a slight alteration from that of all the others. A six-wheel base rather than a four-wheel base is not innovative. Using Mechanum Wheels is not innovative. Doing something completely out of the box, such as creating a mechanism that shifts the drive motors to the lift, would be an example of something that should win.

I don’t think that all of the awards should include the Interview Process. Some of them should be based solely on robot performance. Some teams just don’t have time to interview if it’s just a few people.

I actually don’t mind only giving out World Qualifying spots to the Excellence and Tournament Winners. They’re the ones who did the best at the competition, they’re the ones who deserve to win.

I disagree with the Design Award being a World Qualifying Award. If you’re going to award teams solely for having an amazing design process, you have to give equal weight to the programming process of the competition, meaning the Amaze Award should also be a qualifier if this is. The Design Notebook is already necessary for Excellence anyway.

I think there should be an award for teams that have a unique approach to the game. We see hundreds of designs that are all the same, but there’s something to be said for having an original idea. This is assuming the design achieves its purpose, and isn’t just a bad concept.

There should also be an award for playing smart. I have NO idea how it would be judged, but an award for a strong, strategic approach would be fantastic.

I think there’s too much emphasis on the Design Notebook. It’s great to encourage teams to keep track of what they do, and the new notebooks are an example for teams to follow that don’t know what they’re doing. But I’ve seen teams with 600+ page notebooks for robots that are only okay. Instead of spending so much time making that notebook, it seems like they should have been rebuilding and reprogramming their robots.

We rarely have time for interviews. Even with 10-12 people on the team, there’s not always time to talk with the judges. The few time we have been able to sit down for five to fifteen minutes and just talk about what we’ve done this year have been fantastic, and we’ve gotten great advice. I’d definitely say that they’re a good part of the competition experience.

3

Okie dokie.

I’ll skip to number 7.

7. What has been your experience with the Interview Process at local competitions?
At NZ Nationals last year the judges were a bunch of guys from a few companies, like Rakon, SMC, and Autodesk. They let you show your design folder/notebook and they asked a few questions. This year, we had a couple of guys from Massey University who had already read through the folders (we handed them in the day before), and then they asked a bit, but mainly gave feedback and ideas for improvement. A lot of the teams didn’t really want the feedback, and found it hard, but I quite liked it. The judges were choosing a team to win the award and qualify to go to the World Championships, so they wanted New Zealand to have a strong showing and gave us all feedback to help us.

I think the Design Award should be left as judged on the design process. Design/Designing/Designed/Designer - they all refer to the planning and the process beforehand, and during. Not about how the robot looks/performs. That’s the build award. That is my opinion anyway.

My main view on the judged awards is that there are rather a lot, and some of them seem similar to others, just with slightly different descriptions and of course a new name. I can see how this can cause confusion, and think that it would be better to have fewer awards. Personally, I think about 5 judged awards should be the most. This way they can be all more specific, and less confusion as a result.

~George

4

I think the criteria for the awards is spot-on, but I have been to a couple of events where I had to wonder what the judges were thinking, and I have been to a competition where things seemed a bit rigged.

Some of the smaller events give the excellence award to the highest seeded team, even if the rode they coat tails of much better alliances to the top. The event I witnessed this at did not have a single judge that I am aware of, engineering notebooks were not asked for or looked at, and teams were not interviewed. And yes, the 2 world championship qualifying awards went to 2 teams from that school district.

The next award I had seen questionable recipients of was the Sportsmanship award. I have seen teams suck up to judges and treat other teams like poo get it while some shy awkward students were loaning batteries, helping with programming, and even handing out needed parts only to go home empty handed.

As far as the design award I have been impressed by most of the teams I have seen win it, I did not witness it being a gimme award at any of the events we went to. At the larger events the Excellence award winners were tops in programming and robot skills, and were very competitive during qualifying rounds. Another thing to keep in mind is picking the person who is best with the judges, my son seems to be able to play them like a violin, and I have seen more capable roboteers totally bomb interviews. Pick the most charismatic and passionate person on the team to deal with the judges and that makes a difference. The clue to winning some of the judged awards is to make your passion contagious when you talk to others about your robot and your program.

In short, I think if the criteria are followed there is no issues with how the awards are given, but the fact that there cannot be a RECF rep at every event there will be some unfair bias at some of the events although I think this is the exception and not the rule. There is one event we will NOT waste time going to ever again, but given the fact that we went to 9 events this year I think that is not a bad average.

If you want to make sure everything is perfect every time then that would require an RECF official at every event which would mean that we would have 1/2 of the events to go to. With biased events being rare I don’t see this as a big issue. Considering the rate that VEX competitions are growing at I am amazed that VEX and the RECF have been able to do such an outstanding job. I think that if concerns are voiced through the right channels in the right fashion then they will be resolved. The thing that nobody will ever be able to change is human nature, and that will always effect the outcome of some matches. We have never felt shorthanded as far as awards go, and did not pay enough attention to many other robots and teams to see the finer points that the won awards with.
**
If I could change anything it would be to require that robot skills and programming skills be mandatory for events to award the Excellence Award.**

5

The Design Award is the Only award that requires an engineering notebook. The Excellence Award can be won without an engineering notebook although at larger events it is really needed to be competitive amongst the better teams.

Really? What do these 10-12 people do with their time? We are a 1 student team and have never missed an interview, and we have gone to some large busy events.

6

We have 3 people per team driving. That’s got them either constantly driving and practicing with or fixing the robot. There’s 1-2 people in the stands at all times scouting. On the off-chance that the judges catch one of us in the pit, it’s almost always on the way to do something. I’ve missed coaching a match because I stopped to talk with judges, and gotten yelled at for it. At the competitions we typically attend, there are so many fields running that you’re almost always either in a match or on-deck for your next one. There’s very little down-time other than lunch, when the judges are also eating.

7

I still think that things are pretty good as they are and with a little tweaking they could be better. But we also have to consider that we are not in a position to see things from the perspective of the RECF, some things might make perfect sense if we knew more about the rational behind the award or criteria.

8

  1. First I would take teamwork out of the descriptions as that excluded me along with 2 teams from the world champion alliance. As it should be even more amazing that the team preforms so well with only 1 person. But even if the wording gets changed. Also to me create and innovate seem the about the same…

  2. Amaze.

  3. Design as this is basically just a runner up excellence award in my opinion.

  4. What needs an award is… maybe a comeback award. For teams who start a tournament badly because their robot broke and by the end they improve greatly? I’m not sure on this one…

  5. To heavily weighted is the interview. Its like they are giving awards for who is best at being interviewed. Judges should have more emphasis on watching matches. Very rarely do I see them watch matches. Also they should get rid of judges award…

  6. Personally I think judges put way to much on the interview. And rarely do I see them watch matches. Specifically a robot got build award at a local championship even though they tipped over in 2 matches… its like they believe everything you speak. Someone else got think award but only ran an auto once or twice. And the interview is more of a measure of your ability of being interviewed. Maybe your not good at that or shy or need to fix your robot (cause your a one person team) but deserve the awards??

I think that’s it…

9

After a little thought I decided to post 1 little question for people to think about in here and then not post here again because everyone will look at my opinion as biased even if it isn’t.

First goal of the Vex world championship which is it?
Is it to create a series of positive reinforcement to better encourage teams to use and learn engineering principles for the betterment of students in STEM related topics as well teach students about setting a goal of success to strive for.

Or is it to put together the best robots and best teams and best designs to compete for the title of world champion.

Obviously which ever it is the other one is still very high on the list of priorities. Vex is an educational platform so they care a lot about education but it is called the Championship so which is the first goal.

I don’t have an answer to this that satisfies me but I think that if everyone understood that there are 2 goals fighting for the top rather than just the 1 they care about we would have a lot less complaining.(my own included)

10

I think your on to something, but rather then being “to create a series of positive reinforcement to better encourage teams to use and learn engineering principles for the betterment of students in STEM related topics as well teach students about setting a goal of success to strive for.” or “to put together the best robots and best teams and best designs to compete for the title of world champion.”, I think the number one priority for the world championship is something more like, “To create a series of positive reinforcement to better encourage teams to use and learn engineering principles for the betterment of students in STEM by gathering the most competitive teams in every area in order to create one large learning environment.”

I think this is the main idea of the event because, no matter how you slice it, the main goal is education. If you only got all the teams that were amazing at building, programming, and competing, no one would be able to learn about the best documentation in VRC. This is important for a program that reinforces STEM education, because this kind of documentation is required in many, if not most, STEM careers. VRC is a way to get students some hands-on experience in a “real-world” situation. This is why the interviewing is so important as well. Interviews are required to get jobs, sell products, gain investors, and much more. If the world championship excluded the teams that were best at this side of the competition, we would learn less and the event as a whole would be less awesome!

In regard to the problem sbdrobotics noted with the sportsmanship award, my area doesn’t have this problem. I never knew that the sportsmanship award was a judged award in other areas! Here, it is voted for by all of the teams. Maybe that is something you could pitch to event organixers in your area.

11

While interviewing certainly is important in the professional world, I really don’t believe that the world championships has any significant focus on interviewing. While excellence is supposedly the most prestigious award, just as chairman’s is in FRC, the competition is still about the performance of the robot.

12

Bingo!!!

13

The awards are based on the performance of the robot to an extent, but all of the awards at worlds are very heavily based on interviews as well.

14

I think the interviews for judged awards are very important. I know of several schools where the teacher or mentors built the robot and the students only drove them. Doing the interviews sorts through these teams and culls out the teams that really had little to do with the design and building of the robots. I was rather impressed by one robot, and when I was asking questions about their use of sensors for what was a very impressive autonomous program only the adult was able to answer, the kids had no clue!

Interviews also give teams with a bad match schedule a chance that might not have been possible if the judging was based on the leader board alone. As for the matches we went to this year The Texoma Vex Regional and Battle by the beach had judges intensely watching matches as well as chatting with the teams. With the exception of one that I question all of the tournaments we went to were well done, even three that were their very first time to host a match.

I do (in my own opinion) think that the Excellence award should not be given at events that don’t at least have other judged awards. If you look at the scoring sheet for the Excellence award it has a place for points given based on robot and programming skills as well as consideration for other judged awards, with only match performance being the criteria for the award it does not reflect the true meaning of the award.

But I do like the judged awards and really don’t find the criteria vague for them. As for the sportsmanship award only around three of the matches we attended requested input from other teams, the rest were determined by the referees and event personnel who most likely only witness the teams on their best behavior.

15

Yes, my point is that the biggest part of the event is definitely robot performance, not judging. I feel like this is somewhat in conflict with your idea that “no matter how you slice it, the main goal is education.” Teams that go to worlds might pick up some tips and tricks for robot building and driving, but there are probably very few teams that learn how to make an engineering notebook or learn how to interview.

16

This thread is very informing as it gives us an insight into how teams perceive the awards.

Please consider the following when thinking about how awards are judged.
The world is a very big place and it is very hard for normal people to get noticed. The competition for jobs once you leave school/university is immense and you need to have something special to offer in order to secure your dream job. Employers look for people that are team players and who give more back to society than they receive from it. A person that excels but does not work in a team may very well be overlooked for a less capable person who is obviously a team player as, in the long term, teams of people with succession plans will outperform isolated individuals.

When judging at our events we look for the teams that stand out above the crowd for one reason or another and then we look deeper into what makes them stand out. We realise that teams put an amazing amount of effort into their seasons and we use the awards to recognise that effort.
I real life you may have the best product on the planet but you may only get three minutes in a lift to ‘sell’ your idea to the investor you need to be able to develop it. Yes this has happened to me. Think yourself lucky to get 10 mins with a judge and plan accordingly. If you are not in the right place at the right time you lose. That is the hard facts of real life as well.
Judges often observe a team from a distance to see how they work when not being interviewed. You would be astounded what is observed that way. A mentor or adult diving in to fix your robot once the judges have left may not be the best idea.

Awards are NOT handed out like candy and every single one is earned.

As an engineering employer of many years I am pleased that the VEX Design award has the focus that it has. Engineers that have no idea how to document their thought process are hard to work with and I will not employ ones that cannot show evidence that they can. Randomly building and modifying robots may work but you are leaving no legacy behind you that explains your thought process for others to learn from. Without an engineering notebook your efforts are wasted in the long run and will blow away after you leave the team like tumble weed. You will (I guarantee it) find yourself trying to remember why you decided to do something one way and not the other and having to go over ground you have already walked over. What a time waster!

In New Zealand the Judges Award is regarded as one of the top awards as the team that gets it was a top contender for several awards and the judges feel that they need special recognition.

Our major sponsors have agreed that a team cannot be made up of one person and as such we cannot ever see a team-of-one ever winning the Excellence Award in New Zealand. Teams-of-one indicate that the individual has not worked out one of the very fundamental essences of engineering yet. At the very least we look for a team that has a succession plan (students across many years directly or indirectly involved) and that the team embraces the whole of the VEX competition from robots to on line competitions and mentoring/promoting the competition in other schools. It is entirely possible that a school can/will win the Excellence Award in successive years.

I hope this gives you some constructive food for thought.

17

Some one member teams are that way because there is not more team members available. Our Sack Attack notebook included the recruiting efforts we made, we even offered to buy the local school district a robot kit and had no interest. I think it is wrong to exclude teams of one unless they are from schools that have several teams of multiple students and one chooses to work alone. I agree that collaboration is a huge part of engineering though. We are doing a robotics summer camp in July and hopefully can get some more members, but if we remain a 1 member team it’s not due to lack of trying.

I think before single member teams are excluded the circumstances should be evaluated first.

18

I don’t really think you can say performance plays a bigger part then the judging or the other way around. I think they go together more well then you would think. It isn’t likely that you will get awards at worlds withe a bad robot and good notation, just as it isn’t likely that you will get awards with a good robot and bad notation. There may be a small amount of weight on the performance side, but without a good balance of both, awards are going to be sparse.

If this statement is true, which it may very well be, it is only because teams aren’t looking for tips on documentation, not because there are none to learn. I know that my team and I have learned tons from other teams in terms of documentation. At worlds and just at local events. I know at worlds, one team showed us how they did a CAD model of a robot, then took every piece and put it on a parts list, then used this list to find the most cost effective way to buy parts from the vex store. This seems simple enough, but many, if not most, teams over look this and just buy any parts they think they may need. There were also a few teams that were giving out copies of their vex CAD libraries on cd’s. There was one team showing me specifically how they formatted their typed engineering notebook. There were many teams that were showing me stuff about their websites and such.

This being said, there is obviously quite a lot you can learn from other teams about their documentation methods, whether a team actively seeks it out is up to the team. Unfortunately many teams probably don’t because they don’t think documentation is as important as it is. I know my team didn’t until we realized that that was the reason we never got any awards.

19

To solve the issue of 1-person teams being judged on teamwork, the rubric should be something that can fit everyone, like this:

Criterion for Teamwork:
All members of the team work well together and do not interrupt each other in the interview.
Judges’ Evaluation:
Well this kid seems to work well with him/herself and does not interrupt him/herself in the interview, so this team has passes the teamwork portion.

Teamwork for one-person teams can also be evaluated by observing how they interact with alliance partners (after all, the teamwork element is the whole point of alliances, right?). If someone can adapt and work with people they’ve just met, then surely they’re a good team player. If they are bossy and mean to their alliance partners and yell at them all the time… then they are less deserving of an award that recognizes teamwork.

20

I have a question about the awards that is unrelated to what the original question is, but I feel it could be answered here. in my region there are only five different awards that are given out the Excellence, tournament champions, finalists, skills, and autonomous skill why does our region only use these awards and not include the others? :confused: