Should Skills count more than Design?

VEX Robotics Competition Discussion VRC
#21

My whole thoughts on the design award changed when I saw a team from my state win the design award at Worlds with a Clawbot. And I mean barebones VEX IQ Clawbot.

I go back and forth on the documentation -> effective robot argument. I’m normally of the belief that if a team really followed the design process their robot performance would be better. Then again, some things don’t work in engineering, and you reevaluate and start the practice over.

I guess part of me really dislikes how we entirely separate the robot aspect from the design process. It doesn’t feel organic in that way.

11 Likes
#22

That is a good summary of the EP Summit comment. I think one area would be better coach training about Engineering Notebooks and Design Process in VRC. Really, the feedback should come from team peers and coaches - competition is where you show your stuff, not a training session on what should have been a season long endeavor. Don’t get me wrong - I encourage judges to give verbal feedback and praise on one or two things they saw as positives. But getting thorough feedback is what should be done on the team’s home front.

As for consistency for judging at events, still work in progress. I still go to events where it is not done correctly, but the volunteers who jump in for all these competitions are doing the best they can with the information they have. I think EPs, Head Referees, and Judge Advisors are the key elements to well run events that are consistent with what is to be expected at Worlds.

1 Like
#23

That is not quite true - the judging process explicitly says to observe robots in the field to see if team claims are supported by implementation. This is one of the reasons judging should not be done in a room away from competition. Best practices include going to observe teams at pits, before/during/after matches and accounting for all team members in the interview.

1 Like
#24

The world of engineering is about trial and error. Often you fail more than you succeed and it is important to accurately and thoroughly record the process that led you to the best design. Below are the criteria for the Design Award. It is clearly not about having the best robot but about the design process. In fact, the definition of the word design is: a plan or drawing produced to show the look and function or workings of a building, garment, or other object before it is built or made.

Winning the Design Award requires not only a well organized engineering notebook but a team that interviews exceptionally well. As a professional in a STEM field you must be able to effectively explain your design process, your failures, and how those failures led you to the next step.

As an adult who has experience in VEX and in the world that comes beyond high school, I strongly feel that mastering the criteria needed to win the Design Award prepares you extremely well for the real world - and that is the entire point of what the REC Foundation is trying to accomplish through robotics.

I mean, NOTHING, gets built or manufactured without first undergoing an extremely thorough design process. Design is literally the key to every successful product. No reputable company in the world will let their employees build anything without designing it first.

Also shown below are the criteria for the Build Award. This award does not result in a Worlds slot in every situation (it does in some states) but it clearly rewards a team with an exceptionally build robot. This award is about the final build and not the process of building.

Design Award
The Design Award is presented to a team that demonstrates an organized and professional approach to the design process, project and time management, and team organization. The team’s Engineering Notebook and Judges interview will demonstrate the team’s ability to produce a quality
robot with minimal adult assistance.

Key criteria
• Engineering Notebook must be submitted (usually at team check-in).
• Engineering Notebook demonstrates a clear, complete, and organized record of the robot design process.
• Team demonstrates effective management of time, talent, and resources.
• Team interview demonstrates their ability to explain their robot design and game strategy.
• Team interview demonstrates effective communication skills, teamwork, and professionalism.

Build Award
The Build Award is presented to a team that has built a well-crafted and constructed robot.

Key criteria:
• Robot construction is of high quality; robust, clean, and effective use of materials
• Robot efficiently uses mechanical and electrical components
• Robot is designed with a clear dedication to safety and attention to detail
• Robot demonstrates reliability on the field and holds up under competition conditions
• Students understand and explain how they worked together to develop their robot

6 Likes
#25

Think here lies the contradiction between the end goal of design award (I.e. ability to produce a quality robot with minimal adult assistance) and the key criteria?
But to be fair, I have not seen the judging rubrics before, but based on the listed key criteria, clearly quality robot is not part of the judging criteria.

6 Likes
#26

Here is a link to the Design Award rubric. Nowhere in the rubric is the performance of the robot addressed. It is all about the design process and the ability to explain that process.

https://www.robotevents.com/season/130/document/71/download

#27

But that’s what I mean about the contradiction - in the writeup about design award, the end goal is that the team should have the ability to produce a quality robot.
But this “quality robot” component is not listed in the judging rubrics.

6 Likes
#28

Quality robot is Build Award, not Design Award.

1 Like
#29

@Gear_Geeks

I am looking at the writeup of design award - the last statement.

3 Likes
#30

Got it.

(not a robot)

1 Like
#31

I think it is worth keeping in mind that part of the design process is to develop a strategy and then design and build a robot to execute that strategy. At least, that’s what I teach.

This is an IQ example from this year, but it should be illustrative… The team looked at the game and decided that the way to maximize the teamwork scores was to focus on scoring red and blue cubes, on the assumption all their partners in teamwork were likely to be focusing on green cubes. So they wanted to be able to carry 2 cubes across the field. They also set as a requirement to be able to carry the 2 cubes while driving forward, because their team was all inexperienced drivers and they thought driving in reverse too much would be hard. They ended up selecting a design that was 2 independent 4 bars on the front of the robot. It best met their strategy. They then executed that design, and documented every bit of it, including SnapCAD drawings. They made a plan that have the robot ready by their first tournament.

They came in last in their first tournament. Turns out that a robot trying to carry 2 cubes next to each other is very unwieldy on a field with another bot. It also doesn’t allow for much scoring in the robot skills. They won the Design Award and I think they deserved it.

I understand that teams can show up with bad robots and win Design. It’s also possible for teams to show up with robots they didn’t design, and perhaps didn’t even build or program and dominate the tournament. Look at some of the rule VEX has had to institute… no sharing robots across teams, no sharing drivers across teams. We could question the validity of any award, especially if we aren’t winning it

6 Likes
#32

@ChrisR246 think most of us (if not all) can agree on the need to have EN focusing on the design process, etc.
But what some of us feel is that there should also be a component of the robot performance involved in it as well.

If the team with an amazing EN, but at the end of their design process, decided that clawbot is the best, then clearly there is something wrong with their game analysis and strategy. Then it shouldn’t be given the design award.

If a team comes with an amazing robot, but is unable to verbalise their design process or thinking process on how they arrived at this robot design, then clearly it shouldn’t win design award as well (but maybe be given the build award?).

But end of the day, robot performance or quality of robot should be part of the criteria for design award.

13 Likes
#33

I understand that… I think then too, given the actions that we know have occurred and the goals VEX has, there should be some component of Tournament and Skills winner that satisfies a criteria that the team actually did the design and build, not simply copied something and spent all their time on perfecting driving. That’s not what we want to give awards for either, right?

Perhaps the solution is to eliminate all awards but some form of the Excellence (recognizing that mix of performance and design/planning) in a number equal to the number of qualifying spots a tournament gets

3 Likes
#34

True. since skills can reflect robots and team performance( some times its not true because some team build a robot just to do well in skills and end up bad in matches) , where Design does not. I think the reason why the do this is because they want to promote the engineering design process and want students to use it.

#35

Most of the time… a team that wins skills will have also done very well in matches. if this is not true, then the reason probably has something to do with that team having bad pairings their entire tournament and/or against the toughest opponents.

I think robot quality does have a small influence on design. One of our teams from skyrise year won design at worlds. They had a decent engineering notebook, one that included enough information to explain their robot well to the judges. However what really made team 9364 shine was their team interview. they thoroughly explained the process of designing their intake, and went through the different prototypes they made with physical examples. their notebook touched on all the key components just enough for the judges to give them credit. this was one of the team’s first engineering notebooks, and from what I heard from our coaches, it was quite rushed and makeshift. so really it was the design of the robot and the process of designing the robot that allowed them to win. it was not necessarily the most elaborate notebook.

7 Likes
#36

I think we can all agree that if design Is gonna qual teams for states or worlds, the robot quality should reflect the effort put into the notebook. If we’re thinking about the real world, u could have the best documentation in the world, but if u don’t have results, it practically doesn’t matter

5 Likes
#38

You’ve gotten hung up on the word design - the Design Award seems like a bit of a misnomer due to being a shortened version of what it actually is, a design process, documentation, and presentation award. The award isn’t about how cool the robot is, or how much better it is than other designs, it is about learning and applying the engineering cycle. The teams that are winning Design in places that actually follow the judge guide are not teams who are highlighter-happy, they are teams who have spent often hundreds of hours focusing on the process by which good engineering happens.

5 Likes
#40

I guess perspective on the engineering process depends on your industry. I work in the space industry. So, when we’re developing a billion dollar spacecraft, we follow a rigorous systems engineering process. Some parts on a spacecraft (“long lead parts”) can take years buy - so the design needs to be solidified at a certain point. There have been best practices developed over a number of years, industries and projects and I do believe that the rubric (especially the updated rubric) reflect that. Basically:

  1. Understand your requirements - what does what you are going to build need to do?
  2. Develop options
  3. Select option
  4. Build
  5. Test
  6. Operate.
    (Then iterate)
    Your post comparing Storm Chaser with Virtual Reality headsets) relates to “select option” part of that process. There are numerous ways to enable the selection: modeling and analysis, simulation, prototyping (Virtual Reality), analogy/experience (roller coaster), and BOGSAT (my personal favorite “Bunch of Guys Sitting Around a Table”) to name a few. Just because one project prototyped and another used analogy/experience doesn’t mean they didn’t use the systems engineering - engineering design process. The Design Rubric does not dictate HOW to select the option - only that a team brainstorm options then use a downselect process. From the rubric:

“Explains why the solution was selected through testing and/or a decision matrix. Fully
describes the plan to implement the solution.”

It doesn’t state that team must prototype. It states that you need to do “something” to show how you made your downselect.

Students follow the six steps above whether they realize it or not or put it in the engineering notebook or not. Even a team that goes straight to building a traybot has done a brainstorming and downselect. Based on their experience, they feel that the traybot is the optimum option - they know what the other options are and know their drawbacks - so they brainstormed and downselected. They just didn’t write it down.

OK. Finally, I do have to bite on the “design award is given to teams with more girls” bait. Yes, you are right. But it is not because they are girls and judges like to award awards to girls. Something about teenagers where girls are better at the skills required and have more patience and interest in an engineering notebook. Plus, many teams will relegate their one girl to doing the EN. This is fine and dandy, but in the professional world both men and women must be able to document their work. I am a boss in real life. If I had a male employee asking a female employee to document his work, we’d have to have a talk. So, even though this is happening at the high school level, I would hope that everyone involved in an engineering discipline takes an interest at some point in learning how to document their work.

As an engineer, you can have the most brilliant and creative idea in the world, but if you’re stuck in the lab not communicating with anyone - either through written or oral means - your idea is likely to die on the vine. I believe that the Design Award process is trying to simulate this real-world experience. So, yes, charisma counts! How many monotone briefings have you sat through? Create some hype over your idea! Display some energy! Engineers do not have to be boring and monotone. Just like physicians must also be businesspeople, engineers should also be able to sell their ideas.

16 Likes
#41

Both of your examples, if they documented their work (and I can guarantee you that both did, one for liability and the other for intellectual property/patent protection), would have followed the engineering process. Define the challenge, brainstorm, choose a path, build and manage resources, test, find the failures (defining the new challenges), figure out possible ways to solve those failures (brainstorm), pick an option… you get the idea. Just because TMC didn’t build a prototype doesn’t mean they didn’t follow a basic engineering process (this is effectively the scientific method for making stuff), and what Oculus did is really not that methodologically different.

Regarding VRC (and IQ), how many of these kids have had actual engineering education that prepares them for a professional environment? I would wager it’s a very small percent, that formalized professional development really doesn’t start until college for most and so teaching them the engineering cycle is an appropriate educational exercise. Just like in chem lab and bio lab you aren’t doing the actual work of a chemist or biologist, you are learning the methods and basic principles of the process. The notebook in this context isn’t about actually engineering the best robot, as physics homework isn’t about solving the divide between quantum and astro, it is about learning the process by which that occurs.

The best engineering notebooks in Vex do not all look the same, the teams have not all done the work in the same way, but they have all defined a complete process and have utilized it consistently.

Judging is the thing we can absolutely agree on. These are volunteer-staffed events, and that can lead to really big problems if RECF doesn’t have a handle on what’s happening and how to solve it. Right now the best notebooks in the competition are facing a wall of adults who haven’t read the judge guide, have decided that their modifications to the method are better or easier, and have never judged under a competent head judge. There are teams who are told that what they show in their notebook is too advanced for them, who are working at a level beyond what the judges can process, who are undermined and have mind-games played on them in interviews. That is where Design is failing; not because the award is bad or the rubric is bad or ‘no one uses an engineering notebook in the real world’ (my favorite fallacy), but because there are systemic issues in staffing with volunteers with no consistent training and nothing to demonstrate adherence to the standard accepted practices.

Sidebar - there are great judges out there, who understand the methodology of judging, who are genuinely interested in hearing the teams, who push the judges around them to do better. There just aren’t enough of them.

8 Likes
#42

All of this. Every single word. I cannot click the heart enough to demonstrate my enthusiasm for this post.

1 Like