Although you posted this in the “Official answers” forum I wanted to comment so am quoting you here. There continues to be skepticism about the stability of the IMEs and software controlling them. You have posted about this issue before, although at that time you had not implemented any code for the IMEs. Here is a link for reference.
My current position is that there are no issues in using the IMEs, this is based on working with several IMEs in conjunction with ROBOTC for the last few months, however, it’s not beyond the realm of possibility that both the way I have programmed the IMEs as well as the way the developers test them is different from your configuration. In an effort to help solve this mystery I would ask for your help in the following way.
Please post pictures and a description of your robot so we can see how you have used the IMEs. I am particularly interested in how you have installed the wiring from the IMEs to the cortex.
Provide details on the exact firmware and ROBOTC versions you are using, if possible also provide the cortex hardware revision that is printed on its underside. There are some known issues with older hardware.
Please post (privately if necessary) your ROBOTC source code so I can try and reproduce your problem.
I’ve sent you a PM with a link to the code our team was using. At this point I don’t have an absolute tie in to the use of the IME’s but since 2 other teams at the recent Colorado tournament were reporting problems I asked them to post details, hence the post.
In our case we are using Robot C 3.54 with master firmwarefor Cortex & joystick 3_23 RobotC firmware 0955
Our experience was that after re-engineering the Arm mechanism to have compound gearing we found that from time to time (can’t correlate any specific activity to the occurrence) the robot would lose link (Robot Light on the Cortex was solid red, and then in a short while (not immediately) the Arm motors would spin shearing off the teeth of a set of the smaller gears. This has happened 3-4 times with the loss of 8 or so small gears and subsequent repair time. After much scouring of the user code and a few additional safety checks we disconnected the IME’s and removed them from the code definition. We had one successful practice day and successfully competed all day Saturday with no link loss, no user program crashes where the robot went red, and only one incident of broken gears but that was in the pits and was, in my opinion, a manual fault, There certainly were no link issues.
The code I sent a link to represents the code prior to removing the IME’s The Autonomous routine is commented out as we had not got to testing it yet. so there was no autonomous code running.
The program uses PID control for the arm as there are 2 degrees of freedom and the 2 mechanisms (Shovel Lift and Shovel) have a series of preset locations Shovel lift desired (SLD) and Shovel desired (SD) which are set by pressing buttons on the second gamepad.
The software is subdivided into The main code (competition template) a global variable file, a gamepad task for all gamepad inputs (with one exception) an LCD display section which also takes gamepad input to change PID coefficients if the proper #define is set.
The Motors are controlled from the Main Loop either under PID control or Manual (via calling a function) I don’t believe there is any dual control going on.
we will probably try encoders again, but only after ensuring we won’t break anything.
Oh the Cortex is one of the NC1 scratched out and replaced with a NC2 i.e. one of the ones IFI had upgraded prior to shipping from their description
Sounds like the PID went crazy, I do see some rather unusual ways of doing things so in the words of my old Sony colleagues, “we will study positively”, but probably not today, it’s a busy time of year.
Yes, The PID is my leading contender, but the root cause seems to be something related to the IMEs at least from the first level observables, When IMEs are plugged in and in the PRAGMA statements are the only times we’ve seen the Cortex ‘fail’ which then resulted in the broken gears.
So our potential logic errors aside what is causing the failure in the CORTEX user code If I am to believe the LED indicators on the Cortex and Joystick?
We only have one battery no power expander, so it is not a situation as you previously described with the Cortex losing power and the PID losing the Sensor Feedback.
The PID is referencing a POT, so it seems unlikely if the power is sufficient to drive the motors the POT would be sending invalid values. It could be a case of the PID being sent a value that is not valid, but I’m not sure how that would occur at least in normal conditions.
I’m going to try to get into the school before the holiday and grab some gear for testing over the break.
I have not seen this type of crashing since my Cortex was upgraded from an NC-1 to an A3 but our school has 4 of the NC-1’s that were upgraded before shipment. It would be good to confirm the model type with anyone else having these issues.
Thanks for looking into it, I understand it’s a busy time of year. Happy Holidays
I can confirm that PID was not the common link - at the competition on the 15th, four teams had this issue - us, our X team, and 1069D and E. Our X team was definitely not using PID, however, I cannot attest to the 1069 teams. Even after we took out our PID functions in driver control, we still had the desync problem.
As people have said in previous threads, the field controller and competition switch are no different. However, in a competition setting, there are four robots on the field as opposed to the usual one in a practice setting. This may be a stupid theory, but could this be causing problems?
There is a small difference between the competition switch and the field control system, the field control has one more input into the joystick that forces a WiFi channel change, this was something we learned earlier this year.
I would not assume that teams experiencing connection problems or other issues have the same root cause. It’s entirely possible that software is the problem for one team but a damaged connector the issue for another. I have code from Kevin and also from team 1069E and will initially analyze that as the first step.
I should add we saw our problems just using a competition switch simulator. I started testing with that after the last competition as we had not seen the problem prior to the competition and I was trying to recreate the conditions as close as possible. I should have an opportunity tomorrow night to test some more, and see If I can get it to glitch with the simulator, with and without the PID code.
So far I have not been able to reproduce the issues that 1069E had, I’ve been concentrating on his code as if there is a systemic problem it should show up with either. One theory I had was crosstalk between motor and IME wiring, in an attempt to simplify debugging I have created an IME simulation running on an arduino. The arduino plugs into the cortex and implements the I2C protocol which is on an IME, it creates a running encoder count. The plan is to try and simulate corrupted data and see if I can crash the ROBOTC firmware, however, so far I have not managed to do that. Anyway, I’m not giving up yet.
Thanks James,
I should be able to get the gear from our team tonight for troubleshooting over the holidays and I can continue the investigation from that angle. I’m encouraged by the fact we were able to see the problem without a field controller.
I have a new theory now for our robot. I found the battery that was plugged into the Cortex had a loose connection. The loose connection caused the Cortex to act very flaky and even turned off and on several times. I swapped the Battery and luckily the next battery had a solid enough connection to not cause any problems. I ran the bot for quite a while and did not experience any problems / powered off and on, tested with the field simulator. Next task to test our various batteries to see if any others don’t work well with the particular Cortex. I tried the questionable battery with another Cortex and had no problems. So it’s a specific pairing issue with those two.
Not much progress, the IME simulator works well but my attempts to corrupt communication has not crashed ROBOTC, a good thing I guess.
There are a couple of other observations to mention, nothing new but they are worth repeating.
If you cause ROBOTC to throw an exception the task causing it will be suspended. For example, this line of code is invalid.
sprintf(str,"%w", count);
This causes an exception due to an incorrect format string. When this happens any motor that has been sent a command value will continue to run at the set speed, the competition control still works, if the robot is disabled then the motors are also disabled, but remote control is lost if the task is handling controller input.
If you cause ROBOTC to crash (which is really easy if you know how, several ways, most only two lines of code) then motors will also run at the speed that was last set. The worrying thing here is that the competition control now does not work, I have no idea why as this should be a master firmware function according to the ROBOTC developers but I can only assume that somehow the master firmware has been blocked somehow. It has occurred to me that this is more serous than just an out of control robot as it could potentially be used to give a competitive advantage at certain stages of the match.