IME Freezing

[Jake_Rampertab]

Hi everyone,

I posted a thread earlier about the IMEs: https://vexforum.com/t/imes-becoming-unresponsive/23402/1

After some testing in RobotC V3.60 they have been working fine until recently. It seems that they are freezing at zero. This could easily be caused by a shock or a reset in the cortex. However, I was informed they would only be set to zero and then continue incrementing form there if they were turned. This is not the case for our robot.

We tried a couple things. We recompiled code, added a sensor reset to the code (ie. SensorType] = sensorNone and then back to I2C quadratic…), and a hardware reset all separately. Only the reset fixed the issue. We have also only noticed this happening when the robot is plugged into a competition controller and we change from disable to autonomous. Just some extra data for you jpearman. Hope it helps and if anyone comes up with a solution, not involving a hardware reset, it would be greatly appreciated

[jpearman]

Jake

I’m not sure what to tell you, I have not seen this behavior before. You may have faulty hardware, that’s always possible, but there is no way to tell without some serious debugging.

Does this happen every time you switch to autonomous?

Can you share the code with me?

[Jake_Rampertab]

No this is not happening every time. We have so far seen no patterns in it’s occurrence.

We could however, the code was changed from our competition code to a maintenance code to see the variables and attempt to fix it. For both of the pieces of code the variables stayed at 0. Furthermore, when the robot was disabled and we rolled it around the field by hand the value still stayed at 0. So even when there is no code running, the value stays at 0.

So far it seems like only a power cycle will fix it for some reason. But what does power cycling do differently to the sensors than our code reset did? The code reset was:

SensorValue[topLeftEnc] = sensorNone;
wait1MSec(1000);
SensorValue[topLeftEnc] = sensorQuadEncoderOnI2CPort;
wait1Msec(2000);
//This comes from the reset of the gyro sensor reset I read about last year

As well, we have resently sprayed our field with anti-static spray. So static discharge should be eliminated as a possiblity. We are also running 3.6 firmware and it works very well (Thanks for the advise btw). Any cutouts, resets, or static discharges are being caught and connection is re-established. We have also looked at IMEs after these occur and they work afterwords as well.

[jpearman]

@Jake_Rampertab

No this is not happening every time. We have so far seen no patterns in it’s occurrence.

We could however, the code was changed from our competition code to a maintenance code to see the variables and attempt to fix it. For both of the pieces of code the variables stayed at 0. Furthermore, when the robot was disabled and we rolled it around the field by hand the value still stayed at 0. So even when there is no code running, the value stays at 0.

So far it seems like only a power cycle will fix it for some reason. But what does power cycling do differently to the sensors than our code reset did? The code reset was:

SensorValue[topLeftEnc] = sensorNone;
wait1MSec(1000);
SensorValue[topLeftEnc] = sensorQuadEncoderOnI2CPort;
wait1Msec(2000);
//This comes from the reset of the gyro sensor reset I read about last year

As well, we have resently sprayed our field with anti-static spray. So static discharge should be eliminated as a possiblity. We are also running 3.6 firmware and it works very well (Thanks for the advise btw). Any cutouts, resets, or static discharges are being caught and connection is re-established. We have also looked at IMEs after these occur and they work afterwords as well.

I don’t think you can reset an IME in the same way as a Gyro, I will try that tomorrow.

So lets back up a little, here are a few more questions.

What color are the leds on the IMEs when this happens?
What rev cortex do you have?
Have yoy tried swapping out the cable from the cortex to the first IME?

When this happens again, remove the 4 pin IME cable from the cortex and replace it (carefully), see if that allows them to reset. It feels like a bad cable or perhaps bad IME to me, I have no other explanation because I have never seen this, usually they always reset to zero and then start counting again.

Another thought as you are trying debug code, add this global variable to the beginning of your code.

TI2cStatistics i2c_stats;

and then update that variable in a task like this.

#pragma config(I2C_Usage, I2C1, i2cSensors)
#pragma config(Sensor, I2C_1,  ,               sensorQuadEncoderOnI2CPort,    , AutoAssign)
#pragma config(Motor,  port1,           motorA,        tmotorVex393, openLoop, encoder, encoderPort, I2C_1, 1000)
//*!!Code automatically generated by 'ROBOTC' configuration wizard               !!*//

TI2cStatistics i2c_stats;

task I2C_Stats()
{    
    while(1)
        {
        getI2CStatistics(&i2c_stats, sizeof(i2c_stats));
        wait1Msec(10);
        }
}

task main()
{
    StartTask( I2C_Stats );
    
    while(1)
        wait1Msec(10);
}

Then when the IMEs freeze take a look at the globals window in the debugger, it will look something like this.
[ATTACH]7413[/ATTACH]
If you can get a screen grab and post it back, I’m interested in seeing where the errors are, message being sent etc.

[Jake_Rampertab]

I thought that code resets the sensor by re-initializing it as a quadratic encoder.

I don’t believe we checked the lights.
How do we tell what rev our cortex is?
I will ask my build team tomorrow morning if they did change the wire or IME.

If we can replicate the problem, I will get that data for you. Unfortunately we haven’t seen this problem reoccur. Different from the past, we are writing our autonomous code tethered. We will go back to using the vexnet keys tomorrow. This may make it occur more often. We have seen the error occur more so with the keys than tethered.

If you will be at worlds and have a chance come by our pit. My coach and I would like to thank you in person for your help and hopefully discuss this error in person.

[jpearman]

@Jake_Rampertab

I thought that code resets the sensor by re-initializing it as a quadratic encoder.

It may, I don’t know for sure but I didn’t think so.

I don’t believe we checked the lights.
How do we tell what rev our cortex is?

The rev is on the underside of the cortex, it will be something like NC1, NC2, A3 or A4. It the cortex is on the robot it will be hard to see so perhaps just say if it has a flat top or a raised bump for the VEXnet key. Older versions of the cortex (NC1 and earlier) I think had a problem with the I2C port, there was some discussion of this about a year ago but if it is rev NC2 or later it should be ok.

I will ask my build team tomorrow morning if they did change the wire or IME.

An easy fix, many problems can be caused by bad wires.

If we can replicate the problem, I will get that data for you. Unfortunately we haven’t seen this problem reoccur. Different from the past, we are writing our autonomous code tethered. We will go back to using the vexnet keys tomorrow. This may make it occur more often. We have seen the error occur more so with the keys than tethered.

Sorry you are still having problems, hopefully you can find a solution. Perhaps install the backup battery, some issues can be caused by the drop in voltage when motors first start, when a competition switch is connected then the backup battery will help keep the cpu running correctly if this happens.

If you will be at worlds and have a chance come by our pit. My coach and I would like to thank you in person for your help and hopefully discuss this error in person.

I will try and come by, if not you can find me at the math division, one of the field techs.

[Jake_Rampertab]

Okay it just happened again!

so the lights are all doing single flashes of green except for the last IME in the daisy chain which is doing doubles of green.

Here are the screen shots you wanted are attached.

We have not changed that IME or wire yet but we can now.

The rev is A3.

[jpearman]

@Jake_Rampertab

Okay it just happened again!

so the lights are all doing single flashes of green except for the last IME in the daisy chain which is doing doubles of green.

Well that shows they are all initialized corectly, single or double flash is the difference between terminated (last IME) or not terminated.

Here are the screen shots you wanted are attached.

If these are correct, it shows that ROBOTC is not even trying to communicate with the IMEs, the nTotalPollingCycles and nTotalI2CMessageSent should be counting and not 0. Did you have to connect the debugger after the failure to get this information or were you already in the debugger when they stopped working? When the IMEs are working are all of these variables updating correctly? Did you use a task to get them like I had shown?

We have not changed that IME or wire yet but we can now.

The rev is A3.

Well, that’s a recent cortex, should be ok.

I still have no idea what the problem is.

What other ports are you using on the ROBOTC, can you cut & paste here your #pragma statements at some point.

[Jake_Rampertab]

We downloaded the code you gave me after the error occured. We were testing autonomous code at the time.

Yes they all work well before the error.

I copy & pasted your code exactly.

Here are the #pragmas:

#pragma config(I2C_Usage, I2C1, i2cSensors)
#pragma config(Sensor, in1, ArmPot, sensorPotentiometer)
#pragma config(Sensor, in2, ShovPot, sensorPotentiometer)
#pragma config(Sensor, in3, LineRight, sensorLineFollower)
#pragma config(Sensor, in4, LineLeft, sensorLineFollower)
#pragma config(Sensor, in8, dial, sensorPotentiometer)
#pragma config(Sensor, dgtl1, Bumper, sensorTouch)
#pragma config(Sensor, dgtl2, ShovPist, sensorDigitalOut)
#pragma config(Sensor, dgtl3, JumperLeft, sensorTouch)
#pragma config(Sensor, dgtl4, JumperRight, sensorTouch)
#pragma config(Sensor, dgtl7, SonarLeft, sensorSONAR_mm)
#pragma config(Sensor, dgtl9, SonarRight, sensorSONAR_mm)
#pragma config(Sensor, dgtl11, ArmTouch, sensorTouch)
#pragma config(Sensor, I2C_1, topLeftEnc, sensorQuadEncoderOnI2CPort, , AutoAssign)
#pragma config(Sensor, I2C_2, topRightEnc, sensorQuadEncoderOnI2CPort, , AutoAssign)
#pragma config(Sensor, I2C_3, bottomLeftEnc, sensorQuadEncoderOnI2CPort, , AutoAssign)
#pragma config(Sensor, I2C_4, bottomRightEnc, sensorQuadEncoderOnI2CPort, , AutoAssign)
#pragma config(Motor, port1, Shovel1, tmotorVex269, openLoop, reversed)
#pragma config(Motor, port2, Arm2, tmotorVex393, openLoop, reversed)
#pragma config(Motor, port3, Arm3, tmotorVex393, openLoop, reversed)
#pragma config(Motor, port4, Arm4, tmotorVex393, openLoop)
#pragma config(Motor, port5, Arm5, tmotorVex393, openLoop)
#pragma config(Motor, port6, topLeft, tmotorVex393, openLoop, encoder, encoderPort, I2C_1, 1000)
#pragma config(Motor, port7, topRight, tmotorVex393, openLoop, reversed, encoder, encoderPort, I2C_2, 1000)
#pragma config(Motor, port8, bottomLeft, tmotorVex393, openLoop, encoder, encoderPort, I2C_3, 1000)
#pragma config(Motor, port9, bottomRight, tmotorVex393, openLoop, reversed, encoder, encoderPort, I2C_4, 1000)
#pragma config(Motor, port10, Shovel10, tmotorVex269, openLoop)

[Jake_Rampertab]

As well we think we have found one pattern. This is only occuring when we switch from disable to autonomous in the competition controller. It will happen right at the beginning of autonomous until we power cycle. Still cannot predict when it happens because it happens randomly.

[JohnWatson]

I apologize if you’ve already posted it and I missed it, but would you be willing to post the code or send it to me directly (a PM will suffice if you do not want to make the code public)? I would like to send a copy of it to one of our software developers to see if he can debug the issue.

Thanks in advance!

[jpearman]

@JohnWatson

I apologize if you’ve already posted it and I missed it, but would you be willing to post the code or send it to me directly (a PM will suffice if you do not want to make the code public)? I would like to send a copy of it to one of our software developers to see if he can debug the issue.

Thanks in advance!

Jake (or is it really Jacob?)

If you can please do, perhaps copy me as well.

[Jake_Rampertab]

Jake or Jacob, I don’t mind

No need to apologize, the program has not been posted. At this time we are need ready to release any of our programs to the public. After the worlds, we can focus in on the problem and write some separate code to test the problem with. Our current version is over 2000 lines and not clean. To try and debug what is happening to the robot with all of this on top would be very difficult. If you would like to see it, we have discussed previewing snippets at worlds in our pit. Otherwise, we can release the snippets after worlds.

[jpearman]

@Jake_Rampertab

Jake or Jacob, I don’t mind

No need to apologize, the program has not been posted. At this time we are need ready to release any of our programs to the public. After the worlds, we can focus in on the problem and write some separate code to test the problem with. Our current version is over 2000 lines and not clean. To try and debug what is happening to the robot with all of this on top would be very difficult. If you would like to see it, we have discussed previewing snippets at worlds in our pit. Otherwise, we can release the snippets after worlds.

That’s fine, I could not reproduce the problem you have been having, I suppose that John and I both though that perhaps you were doing something at the beginning of your autonomous code that was unusual or different to any tests we are performing. See you later this week.

James.

[Jake_Rampertab]

I think the only thing irregular is that we start another task to run alongside our autonomous task. besides that there is nothing odd about it.

[jpearman]

@Jake_Rampertab

I think the only thing irregular is that we start another task to run alongside our autonomous task. besides that there is nothing odd about it.

That in itself is not unusual, the non-competition code I write often has 8-10 tasks running that are doing various things. The problem with software is that things that I and the developers may not do you may consider normal, for example, a simple thing that all students seem to do is zero the encoders before using them to drive, this is something I almost never do because of past experience with other types of encoder. It’s not incorrect either way, just different. Anyway, with only two days to worlds it’s probably too late and also inadvisable to make any significant changes to your code.