VEX Worlds 2022 Control System Bug

Just came back from Dallas. VEX Worlds 2022 was super exciting! How’d y’all like it? I’m ready for Spin Up next year. I had a problem tho. Did anyone seem to have any issues with the new field control system there? My team was in the Spirit Division and we lost a couple of matches due to our robot not being triggered. I was wondering if this was a problem for just us or others too. Did anyone else have the same problem and can you share your experience? Do you think VEX should do something for teams like mine who had issues with the new control system? Did the issue get resolved for anyone and how? Don’t want this to happen in the future.

2 Likes

We had an issue with our pneumatics not responding to the toggle commands in our code with the new field control system.

We went to tech support and resolved it by making our setup process consistent, and not selecting the program until we were connected to the field.

4 Likes

Is there a possibility that your controlled was broken somehow?
Like some issue with the original RJ45 control port?

With the new control system, the controller now has 2 possible inputs from the field control, the smart port and the old RJ45, where the RJ45 has a priority if used (or so have I heard). Now, the RJ45 port is rather dumb, there is no communication, just 4 individual pins being pulled to ground to select modes. The pin meaning, when grounded, signal “You are on real Competition field”, “Field is connected”, “Auton mode”, “Disable Robot”.
For the signals to do anything, the “Field is connected” needs to be grounded first, but imagine the following scenario:

  • Having a controller with blown/shorted “Disable robot” pin.
  • Having the new smart port field control providing the “Field connected” condition.

In such a scenario, the robot would still work in the lab with nothing else connected (the “Field Connected” pin not grounded), but might realize the disabled state on the competition field (both old and new). I guess you can test for this scenario using the VEX manual competition switch (276-2335), if you have one.

Other idea, in case it worked sometimes and didn’t the other: Did you always used the same smart port on the controller? (there are two, left and right)
You can also try reproducing and analyzing the issue in the lab if you have a spare brain - VEX published a field control app that works like the one used at worlds, see this KB article.

1 Like

Theoretically the new system would detect shorts on the old RJ45 port, however, there may be scenarios that we could not predict.

We ran 10 division over 5 days and have all data for the approx 5000 matches that were played. The data includes every message sent to and from the new FC system from TM and includes the robot match and radio status. The hope is to analyze all data to see what, if any, problems we can detect. In general we felt the new system performed well. There were some TM operator mistakes, and we did diagnose problems with student code that would have had an impact on their robot performance, those teams would have had the same issues with the legacy control system as well.

27 Likes

We had a few problems with robots disconnecting and also with alliances, we narrowed it down to just having so many vexnet signals bouncing around

1 Like

Do you guys intend to make the data public or release your findings so we can see all the interesting statistics?

2 Likes

I have petitioned VEX for years to make their last software update mid-Feb. This gives teams time to flex it out and make it something for State / Regional Championship.

My view is that the robot is 1/3 mechanical 1/3 programming and 1/3 driver skill. (I float from 1/2 mechanical 1/6 programming to 1/3 driver skill for bots without lots of sensors to 1/3 mechanical 1/2 programming 1/6 driver skill with sensor rich bots. ) But the key part is messing with the programming weeks before an event is the road to ruin.

I’ve done Big Enterprise Development (BED) where one of your “partners” I’ll pick Oracle since I hate them, says “Oh here is a new release”. If we are more than 55% into development , then we use the 'old, crufty, covered with mold and barnacles" version for this release, schedule a compatibility release to be Oracle Only and push back our next feature build.

And while I’d love to slam Oracle, they don’t know my critical dates.

On the other hand, VEX does. They know 2 plus years out my dates for things to be dead tight and working. I love James and the team, I love they are working flat out. I don’t appreciate that their top 8,000 customers didn’t know this was a thing.

I’d like to think that “Foster” is on a list of “that guy runs events, we should beta him” but that didn’t happen. Could it be because RECF has pushed that the lowest level partner is the face, but some of the EP’s are helping them and we miss their help?

We do this EVERY year. Are you updating tax software between 1 Jan - 15 April. Umm no. Can we get a 60 day window in front of Worlds? It’s not like you don’t know 3 years in advance.

Kay this year was a mess, it’s a huge venue, enough wifi to cook a chicken every 11 minutes. But to add in control stuff?

I’m willing to buy into a “it’s a hard cable to the new V5 controller” “it’s a hard cable to the joystick” so we are looking at “wifi only from the controller to the robot”. But I’ll ask, did your beta test include running 48 robots at the same time?

I’m not here to pound a short handed James, and overwhelmed Tim or developers under the gun. I’m hating the release dates when they make a difference.

And I’ll reach out to my fellow EP’s, were you part of the testing for this?

8 Likes

Just as aside, the vendor that provides “electronic hall passes” for our middle and high school, decided to update their software on a Monday morning without releasing it. So of course nothing was working, and guess what - kids could not get bathroom passes as a result. The next couple of days were stressful to say the least. Vendor rolled back the software. Maybe they can try again during the summer. Worst part was the new user interface was awful design.

ugh.

No…

120 robots.

and we tested with 120 each morning before the venue was open.

@Foster your post reads as if we had many technical problems. As far as I’m concerned the system was a huge success, we did identify one or two things on day 1 of middle school that we could never have tested before worlds, very hard to simulate 800+ robots, but there was nothing that was match affecting. We addressed those issues before day 2.

17 Likes

Research division Q1… :sob:

Ok, then I’m good with your testing.

But I’m not good with the last month testing. Oddly, I knew when Worlds 2022 would be in most of 2021. And I think I know when Worlds 2023 will be.

Not sure why RECF wants to jam changes down the month before. This is an ongoing cycle. And while I appreciate, “Make Worlds the Best”, the stress to teams

Slipstreaming in April isn’t the answer.

And I get it, VEX/RECF are not Oracle with millions of customers, VEX/RECF have only 9,000 that really care out of the VEX base of 100,000+ users.

Timing does mater. This corner of my life is a microcosm of my day job with 8 robot teams. 8,000 is a plant site, 80,000 is a region, 800,000 is a company wide disaster, but I’d like to think that I work on the 800,000 level all the time.

Glad that it worked out for most teams @Sylvie may have a different view.

My view: Cut changes after 1 March and get EP to test.

And while I appear to be publicly smacking you, this is really directed at your boss. I don’t know what your reporting is, its so much of a mishmash but I’d like to think that @Bob_Mimlitch_III and @DanMantz reporting trees would think twice.

1 Like

Nothing I can do to stop a rookie TM operator making a mistake, human factor is always a variable.

9 Likes

Anecdata from our experience: 11 matches all seemed flawless from a TM/Control System perspective.

1 Like

For what it’s worth as just another anecdote, PYRO didn’t have any significant issues relating to field control in our matches, and if we did I would have noticed. I’m also not aware of any wide spread issues from the teams that I talked to. Big props to James and Levi for getting the system to be as stable as it was. I do still think that trialing the system at some events throughout the season would have been prudent, maybe it would have helped find some issues that were present, or at the very least it would have helped teams familiarize themselves with the new system before the world championship. Additionally there is a perception amongst the community that worlds is used to beta test new systems and hardware (e.g. bo1, new field control) which could be combated if new systems are used at events other than worlds first.

9 Likes

First things first, mad props to jpearman and his team (and apologies to all other contributors whose names I simply don’t know) for the new field system and generally for all of their contributions to all the cool stuff we get to play with. I always said - big companies with your big budgets and big pyramids: take a look at what can be accomplished with a small but dedicated team of talented people and our no-nonsense iterative approach. I was only able to watch matches in the two divisions my teams were in, for the basic reason that my feet, ankles, shins and knees (v1.0) almost stopped functioning from all the walking. But in those 2 divisions, I have not seen delays or major tech issues, looks like the new telemetry data sent to the brain controlling the field helps a lot (although a bit sad this runs on v5 brains - those are so pricey now and not even available, so harder on smaller or potentially new EPs - and their budgets - who will most likely stick with their Raspberry Pi setups for a while).

That being said, we did experience one issue which we solved empirically. Basically if we turn robot/controller on, THEN plug in smart cable, start code, select auton from our menu, when match starts our auton would start but did not look like anything we programmed (still have no idea where those moves came from). Working theory is that lines of code were skipped. We will examine the live stream footage closer and try to understand it better.

However, if we turn everything on, start program, select auton version from our custom menu and THEN plug in the field’s smart cable as the very last step, everything works nominally. So it’s what we did.

James has an older post describing some of the issues that might happen in a multi-task (or multi-thread depending on your language) environment. We used Pyton/RobotMesh, not a very popular (and not very supported as of late) combo but we stayed away from multi-threading. Our menu loops and runs outside the auton and driver containers but other than that not much complexity. Fun puzzle for the team to figure out.

Thank you RECF&VEX for another awesome Worlds!

4 Likes

This could all be accomplished by building a simple HAT on the RPI and as there is already in most cases an RPI at the field this would be a simpler solution……

While potentially a viable solution on its own, I don’t really see how this would inherently solve any of the mentioned issues with the new system; the software would have to be written either way. Additionally, from my understanding such a hat would require dedicated clocks since v5 smart devices don’t use standard clockgens for their baudrate.

1 Like

There are still Pis under the field to connect to TM. This system just replaces the VEXnet field and match controllers. I had very few issues in my divisions with the control system, and the additional diagnostics really help with troubleshooting.

There were a few hiccups with it, but ultimately we ran over 400 matches in my divisions between VEXU and VRCHS and had essentially no issues.

5 Likes

2/11 matches, my team black screened when the match started. We even swapped radio wire and our brain. So it may have just been a rare error.

Do you mean the V5 lost power ?

3 Likes