I was checking the Chinese VEX Forum the other day and noticed that they have implemented some sort of programming testing procedure for teams for the top 16 teams in qualifying matches, the top 3 teams in skills and other randomly selected teams at select VRC and VIQRC tournaments according to this official post. Here are the documents provided in the original post translated into English using Google Translate:
I found this quite intriguing since this change was only implemented this season in China. I haven’t seen this in person (all the tournaments that I’ve attended were hosted independently and not by the Asian Robotics League), and I’m interested if anyone here has experienced this test and how it was conducted,
I guess this test was implemented to curb mentor-building, ensuring team members have basic proficiency in VEX V5 or IQ. Although this can probably prevent some of the most severe REC Student-Centered Policy violations in the short-term (ex., a mentor entirely built the robot), I think that this test wouldn’t be as practical for teams that violate the student-centered policy but still have some degree of student involvement and that, in the future, mentors could anticipate tests like these in future tournaments and teach team members some topics only because it may show up on tests like these and not for the sake of robot construction or programming.
What do you all think? Although I question the effectiveness of this implementation, I believe that something like this could be implemented in higher-level tournaments in other regions outside of China in future seasons. Still, I’d personally be against it since the judging process at tournaments (specifically team interviews) should be, in my opinion, sufficient to determine the level of student involvement in a team.
I agree with this in practice, but I feel like this would be hard to implement correctly, not to mention the issues that this policy has, e.g, what if the programmer isn’t at the competition, what if the questions are irrelevant to a team’s code and such (though this could be mitigated depending on the difficulty of the questions), or what if there’s an issue with the hardware / software that a team is using (PROS recently had an issue where you couldn’t make a program)
I agree, I think that this test is good in nature but I don’t know how it could be implemented practically in a good way. I do think that this shouldn’t be too hard for the teams that would be required to take a test like this given the skill level at Chinese tournaments.
I’ll see if I can contact some of my friends currently doing VEX in China outside an international school and see what they say.
Yes - this is the main intention.
From my understanding, the team will report to the field, and they will be told to do a simple programme to get the robot to perform certain tasks.
It is just to make sure the students do know their stuff and it is not totally a mentor-bot.
Previous years, there were always complaints about how unbelievably was some of their autonomous/programming skills routines,
Guess USA can consider this approach if the dad-bot situation is really bad.
There is a balance… Telling the students that there will be a test and giving guidelines as to what will be on the test might also make it easier to cheat and get away with it. The mentor builds the bot, students take a new “test prep” course to pass the programming test at the tournament.
Once the “test prep” is more work than the students building the bot, then it will be a very effective technique!
I agree that the RECF should do something to address this problem. Maybe it only needs to happen at regionals and worlds. Now, as I am a first-year VRC competitor, have your salt ready. Here is my idea of a solution.
At large events and worlds, after all teams have been interviewed, go back and interview the top 5 skills teams, the top 5 in teamwork matches, and any team being considered for an award. Ask them to walk the judges through the mechanisms on the robot and display full knowledge of how each part works. Same thing with the programming. These interviews should not be taken into account when giving out notebook awards. (Unless if the team fails of course)
If they fail to display full knowledge, their eligibility for awards should be reconsidered on a case-to-case basis.
I am open to having someone more versed tell me why this won’t work…
Well, to be honest, students studying to learn code is way better than having a pure mentor-built bot with kids clueless as to what a boolean is. Having a test is flawed, but at least the kids are learning something.
You don’t have to give away the exact details of the test, just something like, “section two of the test examines a competitor’s ability to program loops”. However, I am concerned over the varying levels of coding ability amongst teams (how would you test both my team who just modifies the configuration tab code with a few add-ons versus a team who programs a PID/odometry system?)
From what I have seen these “tests” have been around since at least 2021. Here’s a post I made about this from back then.
Essentially, teams within certain ranking thresholds will be asked to undergo these tests. Up to 2 team members will then go to a testing room, where they will have to complete a programming task in 10-20 minutes (2021 docs say 20 minutes; I heard in 2022 this was changed to 10 minutes). The catch is, that no adults or coaches are present for the tests (besides the officiators, who are not affiliated with any teams), and the teams don’t know what the test questions are beforehand. No communication devices are allowed as well, so teams really are truly going into the tests “blind”.
If a team does not pass the test, then they get automatically disqualified for the entire competition. From what I’ve heard, at the 2022 China Nationals out of the 20 teams who did these programming tests, 25% of them failed and were DQed from the event.
Here’s what the testing looked like from that event (photo credit to Helen LLL from Discord); I assume this is similar from event to event.
Speaking for me and just me, I’m going to give this a hard, hard no if it comes from RECF. If EP’s “just don’t have the time to hand back feedback to teams” we certainly don’t have time to administer and grade tests. People think that it’s opening Pandora’s Box to give feedback, it will be opening Pandora’s Shipping Container ™ doing tests like this.
@jpearman I remember for the Think award having teams walk the judge through the code, but Covid changed that.
I should mention that these tests are being conducted at multi-day (usually 3-4 day) events. At these events, the tests are done after the end of qualification matches (usually the end of day 2 of matches) and before the start of eliminations (usually the beginning of day 3 of matches- yes, alliance selection sometimes falls after quals on day 2. Its weird, I know). This gives the event plenty of time to conduct these tests. In fact, for the event I mentioned above, there is a dedicated time slot in the event agenda for this:
(It even says they might need to adjust the schedule based on how long the tests take to administer).
It is my understanding that these tests are isolated to Mainland China, with no real intentions to expand them to other regions (as of this season). This model works quite well, given the unique size and dynamics of their events. I agree that the model would be hard to adapt to local qualifiers in other regions. Perhaps it could be adapted to larger events [Sigs, ERCs, Worlds (I actually heard they might be coming to Worlds; however, this was back in 2022, and I have yet to see any confirmation of this), etc.], but the viability of this has yet to be seen.
Fixed my own quote. Should have put people into it. EP’s are desperate for volunteers:
(scene: White 15 passenger van drives up to Home Depot at 7AM. Driver leans out of the window: “Hey, I’m looking for 6 judges and two queue managers, no pay but a pizza lunch”. Voice from crowd: “Pepperoni and mushroom?” Driver: “Yeah, yeah, pepperoni and mushroom.” Voice:“Coke?” Driver:“No Coke, Pepsi.” 8 people out of a group of 16 raise one hand and walk towards the van. End scene)
I’ve also taught programming at the university level, grading programming tests “write a program that does ZZ” are a pain.
On the plus side, I can’t wait as an EP to forward this email on to RECF.
So yes, still a hard No. Even for pepperoni and mushroom pizza.
Although I do compete in mainland China, the tournaments (ex. the TIS Robotics Challenge) that my teams go to aren’t the typical big tournaments that most Chinese teams would attend so I haven’t experienced the dynamics of such larger events or this programming test. (Because the tournaments I attend are hosted by international schools in China, they are exempted from many of the regulations of VRC and VIQRC in China)
In terms of the tournaments that my teams go to, I don’t think implementing this sort of test would be effective because I’ve seen somewhat of a disparity between the more advanced and newer teams (which could make it harder for newer, first-year teams to compete) and that the events we attend don’t really have that many volunteers, with most of the volunteers being high school students at the host’s school. At the same time, our competition season for teams from my school and others would often only compose of one single official high-stakes tournament (TIS Robotics Challenge for VRC, the WYIS VEX IQ Tournament for VIQRC) with one qualifying spot to Worlds.
This is the test question in 2024 VEX Asia-Pacific Robotics VRC Signature Event, this is so easy, right? But 9123C that is 6th place in HS skill can’t pass.
I would have written “incomplete requirements” and handed it back. How far forward? After the left, just turn right? Is that spin right or turn right. Then start the launcher? How fast is the right turn?
I get that this is a joke example, but having completely clear requirements / functional design is important.
Do you happen to have the original untranslated version as well? I would want to take a look at the original language, too.
And yes, this question seems quite easy for a high-level team and shouldn’t be difficult at all for most teams with sufficient capabilities. As I’ve mentioned, the tests seem to be good at seeing some of the mostly mentor-built teams, but I doubt that this can find any student policy-violating team with at least some degree of student involvement.
I think that was so easy for everyone, they are a high level team in China, but they can’t pass.
They participated in 6 competitions, among which they won one Tournament Champions, one Tournament Finalists, one Tournament Semifinalists, one Robot Skills Champion, two 2nd Place and one 3rd Place.
Coaches seem very useful in China.
Personally, I think this is an actual test judging by the formatting and the language used in the Chinese version. The ambiguity in regard to the distance seems to be intentional, in my opinion and could be even more of an intended simplification of the task by completely neglecting distances for robot movement (unless I’m missing something, perhaps the field was set up in some intentional way?).
Also, the translation isn’t the most accurate and, in my opinion, is a bit clearer in its original form (in regards to the turning part in steps 2 and 3, it asks the robot to turn left 90 degrees, wait one second, then turn right for two seconds). You are still very right in saying that the instructions are quite unclear by neglecting variables such as turn speed, velocity, etc. but I feel like it could be intentionally done to simplify the task.
