Why can't we see a graded rubric?

VEX V5 Robotics Competition VRC
1

Why aren’t we able to be given back our graded notebook rubrics after competitions? I feel like this would be counterintuitive because it creates a blank space where we do not know what we got points off on. This could lead to us not knowing what to improve or possibly changing how we document a certain step in the design process that was previously good as it is.

If the judges allowed us to instead be given a scored rubric with all of the parts that we got points deducted from, it would allow us to know what we are not documenting correctly and what we have done well on. This would help me and many other notebookers because it could show us how to improve our notebooks so that we can earn back rubric points at future tournaments.

2

Hi @Luke36064A!

As a notebooker myself, I’ve often wondered about this and often think it’s unfair. However, it’s been explicitly addressed in the GtJ I believe.

Details of judging deliberations at RECF events are confidential, because teams are discussed and compared. Highly developed teams are likely to score well on the rubrics, so most feedback at an event would be based on comparisons with other teams and would violate that confidentiality.

Scores are inherently subjective and can vary between events.

Well-intentioned feedback from a volunteer judge could potentially be discouraging and harmful.

And much more!

TLDR: The scores are subjective, the constructive feedback could be taken as an insult, and it goes against the spirit of the competition.

Here’s the link to the actual article: https://kb.roboticseducation.org/hc/en-us/articles/23366353239063-Why-Don-t-Teams-Get-Feedback-from-Judges

On the other hand, you could ask your coach and possibly other students in VEX to try grading or give you feedback on your notebook. I’m not exactly sure how, but people on the forum could potentially also give feedback (whether it be on the whole notebook, a certain page, etc.)

Hope that explained it (correct me if wrong). Good luck!

3

I would recommend you read ALL the articles that the RECF provides with regards to notebooks and judging:
https://v5rc-kb.recf.org/hc/en-us/categories/9625651590423-Notebooks?sc=v5rc-design-process-and-notebooks

I would then focus on having people outside your team review your notebook using the rubric. This could be teachers (yup they are masters of rubrics :slight_smile: ), other teams, adults in STEM fields…

You want to focus on how to move towards “Expert” in all indicators on the rubric, both for the adherence to the Engineering Design Process and teamwork and project management that enables your team to thrive with a well designed robot and planning.

It helps to have multiple individuals review your notebook. Some will score harder than others. The key is to look at the indicators and ask yourself what evidence is missing for a criterion for full marks.

Good luck! You will find that a good engineering notebook is one that the team benefits from. If your notebook offers not value to your team, you will struggle at getting better at documentation.

4

Good question IMO

Like @haonp and @lacsap stated, your team or teammates could help you “self grade” the notebook. Using things like the notebook rubric, you or your teammates can evaluate the notebook almost just like how the judges do.

5

I’ve been a judge/JA at a number of competitions as well as a coach/parent volunteer for several years now.

Having been tempted to give even the broadest strokes feedback to coaches/students as a judge from a “learning” perspective, any time I’ve seen a judge/JA literally say anything to a coach/parent/student other than like “no comment,” it hasn’t gone well. Usually it’s a complaint in response to not getting an award, and any additional details tend to be used as grounds for arguing the outcome rather than improving future work.

But I’ve been on the other side of things, where teams shared their notebooks with each other after the season and it became obvious that the that judging process went way worse than “subjectively” and that no amount of improving relative to the rubric would have affected decisions.

From leading coach trainings, I know there are a lot of rookie coaches that just don’t have a good handle on what judges are looking for. They default to encouraging kids to just keep a daily diary or the like because the system provides no strong feedback loop.

This is by far the most frustrating part of the competition for lots of rookie public school kids in my under resourced district- they notebook, turn it in at a competition, don’t win anything, and then are just told to keep doing it. Compare that to robots—students build them, test them, see other ideas at competitions, and then iterate. The notebook process has none of that feedback cycle. Even just suggesting that competitions add more of the available awards (even if they’re not qualifying to states/worlds) to give that encouragement to kids has fallen on deaf ears.

Almost none of our students want to notebook. And after years of telling them about all the benefits, but seeing this cycle, I don’t blame them and I don’t keep nagging. I make sure they’re following the engineering design process, I quiz them on why they made decisions and what alternatives they considered, but filling out a notebook is largely a waste of their time.

So, what do we try to do on the positive side? We’re experimenting with some approaches within the current rules. One thing we’ve tried is to run a league and give out pseudo-awards at earlier sessions- recognizing something like “best Test page” or “best design cycle” or “best technical drawing,” asking students if we can share with the group, and giving some kind of treat as an award. I can tell you this is far more motivating to 90% of teams than anything officially in place. But I’d love to hear other ideas people have!

6

This is a great stance on the problem and I agree with you in that many times we are just told to keep doing the same thing we have done in the notebook. I feel like there could be a way that the judges could give constructive feedback without it sounding degrading or taking away from the ways that the student has documented well. While it does say that giving feedback to students would “violate confidentiality,” I still feel that it would allow for us to grow more and to find a new way to document the notebook.

It is kind of how you said,

I feel like this would be a simple fix by just showing what the notebook scored on each section of the rubric. The rubric isn’t the only deciding factor to judged awards, so I feel as though it would not be revealing too much information about how one team won an award and one did not. In my opinion it would not make sense for their reasoning to be that they do not want people to argue about why they did not get an award, because judges do not need to explain why they chose one over another. I do think that, as I said, the rubric would serve as the feedback cycle, like how a competition is the main feedback cycle for the building portion of robotics.

Please correct me if I said anything wrong or if my explanation didn’t really make sense.

7

One thing I’ll contribute to this conversation, do keep in mind EN19. To summarize:

Developing Engineering Notebooks contain little detail, have few drawings, and are not a complete record of the design process. To save Judges’ time, the Engineering Notebook Rubric will not be completed for these teams.

I’m curious to hear the arguments for how this could be handled if feedback was allowed to be given. Do teams that do not make the Fully Developed cut not receive rubrics/feedback, even if they are the teams with the largest room to grow?

Speaking as someone who has volunteered as a Judge and Judge Advisor at events ranging from small locals to the World Championship, Judges’ time is already stretched very thin as it is. Effectively doubling the number of rubrics that would need to be completed is not a feasible amount of work to throw on a volunteer position. Especially considering the rapid inflation of page-count in recent years.

That isn’t to say I think there’s no merit in giving feedback to teams, I just think it’s a more complicated question than it seems.

8

The question the OP posed was “Why can’t we see a graded rubric?” The answer is simply it’s against the rules, but that’s not really the right question to ask. The better underlying question is “How do we improve our chances to win a judged award?”

I know how to fix this: coaches volunteer to be judges at a competition. EPs are ALWAYS looking for judge volunteers. The BEST way to know how to help your students be more competitive with judged awards is to learn the judging process by being a judge.

Students, you can’t be a judge at most competitions, but if you are looking at how to improve your notebook, ask your coach to be a judge. If they won’t, or in addition, ask a parent or other trusted adult to be a judge. Then harvest them for knowledge.

Also remember that the judging process is not only about how good a notebook is. The team interview is really where the judges identify teams that will be in competition for awards. If teams want to do well in a judge interview, they should practice being interviewed. Not so that they can prepare a presentation and repeat it for the judge, but so they can gain experience on how to clearly and succinctly explain their process and product the judges and respond with clarity and confidence to their questions.

9

Having served as both judge and judge advisor, the big reason to me to not return/show the rubric is there are teams that would argue every single score that isn’t a five…well after the fact. Judging is all 3 things - objective, subjective, and comparative (IMO). Bottom line is, have a good, detail-oriented notebooker who knows the notebook scoring rubric like the back of their hand and you should do well.

10

Reading through this thread, I noticed a lot of “Why won’t the judges do this?” and, while present, very little “What can we do to incentivize it?” As mentioned by others, judges are already operating on a timed schedule, so why would they go out of their way to fill out a rubric for a team if they know complaints are (unfortunately) more likely than the team learning from the feedback? Now this is something we can make a difference in — If we go through the interview process efficiently and politely ask for (NOT DEMAND) some feedback, there is a chance (STILL NOT A GUARANTEE) that the judge will recognize our desire to learn AND have a few seconds to a minute to give a comment or two.

TLDR; Be an efficient and polite interviewee and a judge might just Grace you with some feedback.

11

as someone who judges frequently at competitions, there are a lot of reasons why we cant give feedback and REC has a lot of good reasons on the website and through the GTJ. Mostly, the way we grade is relatively subjective to the competition and most judges are volunteers. Its hard to give good feedback when the people who know little about the specifics are the ones expected to give it. I wish we could give more feedback than we do, but its all a complex process.

There are so many resources on the guide to judging however that give rubrics and evaluations and you can see the areas you fall short by doing practice interviews, peer reviews with other teams on your notebook, and self evaluation based on the rubric and its criteria.

It seems helpful but it would muddy the process and put even more pressure on judges, which we do not need I promise. There are a ton of resources to help you that dont require judging feedback, and that is intentional.

12

I wish there was a way to give your comment +100 :100:

Perhaps, @DanMantz or members of @VEX_GDC have such magik powers.

I wish more people did something like that, trying to help beginners to get various types of feedback from different angles.

It is unrealistic to expect that students (and coaches) who had no previous experience will learn much from the minimal feedback in a form of several numbers, in addition to alien sounding rubric.

If you think about it, announcing two team numbers who won Excellence and Design awards (out of, let say, 32 teams at the competition) is, literally, just 10 bits of information and Notebook weight there is just a small fraction.

Most of the times, if I see that there are teams whose members are eager to learn how to become better builders or notebookers, I would try to spend more time with them during pit interviews, asking questions in a way that suggest how they could improve their skill.

Also, I would bring with me previous season’s notebook from my team to show novice teams sections that could serve as examples for how they could improve both their notebooking and robot design skil.

Finally, for those students who want to excel, it will take for us very little effort and gives great benefit for them to make sure new teams know that they could look up notebooks of the past winners who made them available online.

You can start here or simply create poster or handout with a QR code for a Google search for vex winning notebook.

That will return both Worlds winning notebook examples as well as ton of instructional videos with tips and tricks how to start and maintain top notch notebooking process.