Spurious readings from IMEs? (from EasyC official support)

From thread: https://vexforum.com/t/spurious-readings-from-imes/21863/1
in EasyC official support:

In my experience, this type of thing (“print shows the wrong results”) is more often a problem with print buffering than with the actual data being printed.

For example (not likely real numbers…)
Lets assume the USB to serial port is running at 9600 baud.
At 10 bits/char that 960 char a second, or about 1char per millisecond.
Simple Easy C loops often run about 1 loop per milliscond, (you can time yours).
With these numbers, if you print more than one character per loop, the print buffer will grow and grow.
There has to be a tradeoff somewhere, either overwrite some of the buffer,
or stop the while(1) loop until the buffer gets sent. If EasyC would stop the loop, the robot would drive into the wall while ignoring the joystick/etc while waiting for the print buffer to get sent, so the better tradeoff is to dump/overwrite/corrupt the print buffer.
This makes output data bandwidth an important constraint of real-time embeded systems.

Your hypothesis is that IME is sending bad data.
My hypothesis is that printing too much data makes any data look bad.

Can you invent some tests to disprove or support either of these hypothesis?

Other usual suggestions:
Always post your code, use the strength of numbers (unofficial support)

GregH also posted this same question back in April (yes I have a weird memory).

https://vexforum.com/showpost.php?p=282925&postcount=49

As jgraber suggests, post the code and lets see if anyone else can reproduce this error.

I have also requested, an e-mail with the copy of the code. Seeing the the IMEs poll via an ISR, I wouldn’t think there could be a memory conflict as the poster suggests. As you couldn’t read the data until the ISR returns.

I have been able to reproduce values similar to GregH describes but not with code I would normally expect to see. I suspect that the frequent use of PresetIntegratedMotorEncoder can corrupt values. The code below is not code you would normally run but can cause two bad values to be displayed.

16711780 and -65436

Looking into these values as hex shows them to be

0x00FF0064 and
0xFFFF0064

They are partially written hex representations of 100 (0x64) and could be a classic example of a 32 bit long being written as two 16 bit words that perhaps had an interrupt occur between writing them.

Perhaps the internal storage for the encoder is not defined as volatile.

Perhaps the PresetIntegratedMotorEncoder command runs asynchronous to the IRQ that polls the encoder, anyway, somewhere to start debugging.

#include "Main.h"

void OperatorControl ( unsigned long ulTime )
{
      long x; 
      long oldx; 
      int counter = 0; 

      InitLCD ( 1 ) ;
      SetLCDLight ( 1 , 1 ) ;
      InitIntegratedMotorEncoders ( ) ;
      while ( 1 ) // Insert Your RC Code Below
      {
            JoystickToMotor ( 1 , 1 , 9 , 0 ) ;
            x = GetIntegratedMotorEncoder ( 9 ) ;
            //SetLCDText ( 1 , 1 , "%ld" , x ) ;
            if ( x != oldx )
            {
                  PrintToScreen ( "%ld\n" , x ) ;
                  oldx = x ;
            }
            if ( counter++ == 10 )
            {
                  counter = 0 ;
                  PresetIntegratedMotorEncoder ( 9 , 100 ) ;
            }
            //Wait ( 10 ) ;
      }
}

I had one 393 motor on port9, EasyC 4.1.0.3, firmware 3.21, competition project.

My thought was that this was a multi-threading problem where we were getting a value that was half updated. I wrote a mini test program to try to reproduce the problem and I’m not getting the spurious numbers now. Maybe the spurious numbers were not the real problem, we were adding prints to try to debug the real problems we were seeing which include the following:

  • Cortex sometimes reboots
  • Sometimes our code seems to stop updating, leaving the motors to continuously execute the last command they got
  • Sometimes we seem to not get any changes in the IME values

These problems are reproducing in our test program. The IME’s stop responding at some point and the other problems listed above randomly happen. I added a button to try to reset the IMEs by re initializing and re-presetting but pressing the button doesn’t bring them back. The values just stop changing.

I can’t find a way to copy-paste code from Easy-C so I’ve zipped the project. I’ll email it as well.
IME_Test_Program.zip (2.4 KB)

Greg

I’m running the code now but so far don’t see any issues.

What is the hardware rev of your cortex? It’s on the back, NC1, NC2, A3 or something else ?

If you are not using the IMEs, does other code run correctly on the cortex?

Brainstorming, or rather, more ideas from “the same number of people who work alone and later pool their ideas”.

Per some other IME threads,
GregH could try adding 10ms wait after the first Init before the while(1) loop.

Per jpearman’s hypothesis:

I suspect that the frequent use of PresetIntegratedMotorEncoder can corrupt values.
GregH’s code will execute InitIME and PresetIME every ~34ms if the 8U button is pressed, which may count as frequently.
GregH could try using the rising edge button detector method, or other methods to make sure these get executed only once per button press.

Per jgraber print hypothesis:
GregH could add a loop counter to the while loop, and print the loop count as well as the left and right encoder values.
Then add a if-block around the print, to print only values when abs(val) > 200.
This may resolve the strange values like 6032353 = 0x005c0BE1,
but not the rest of the problems.

Per kingofl337, “IMEs poll via an ISR”, vs jpearman: “could be a classic example of a 32 bit long being written as two 16 bit words that perhaps had an interrupt occur between writing them.”
Can the PresetIME disable interrupts while it is running?
Maybe wrap the PresetIME into a RegisterIMERepeatingTimer routine that only runs once.
Alternatively, avoid use of preset for encoders, and treat them more like a single timer resource: Instead of clear/get timer/encoder, clear becomes “snapshot current value and save it”, and get becomes “snapshot current value and subtract saved value from it”.

Basic question: Do the IME appear to work normally with the On-Line Window? Secure cables firmly, etc.

Answers:

  • other code works fine, we used this cortex for a robot at worlds this year for example
  • It is an NC1 unit that was upgraded to NC2
  • The button logic should use rising edge but that was just an attempt to get the system to ‘reset’. The IME’s stop “changing” even if I never press the button.

You had a good idea. I just tried the “online window” and I see the IME’s stop updating in it too. This is confusing because I keep seeing different problems. I’ll try to be more accurate with this description. Here’s what I saw in the online window:

  • move one motor, it starts counting up and down
  • start moving the other motor, doesn’t count, stays at zero
  • next the first motor encoder stops changing, stuck at its current value
  • tried resetting to zero, now they both just stick at zero
  • tried disconnecting and reconnecting
  • at first no changes, then the second motor ticks from 0 to 1 and then sticks at 1.

I also noticed that the rate at which the numbers are changing is about 1000x slower than I saw them changing in my prints in the test program. On at least one run of the test program, small movements of the motors would generate changes in the order of 1000s.

I’m not sure what to try next. I’m back where we were when we were trying to get these working on our robot (completely different mechanical setup, wiring, and different programming laptop). I just can’t seem to get any kind of consistent behavior.

Try just one IME rather than two to see if it’s a problem with an IME rather than the cortex or software. If one gives a problem then switch to the other and see if you still have the issue.

When was the cortex upgraded? Did you receive it with the NC2 drawn over the NC1 or did you send it in to VEX?

With the online window, Try just one IME on one motor plugged into motor port#1, with that as the only I2C connection plugged into the I2C port.
Note this will require changing the Motor Config panel from your current setup.

This question reminds me of behavior I was seeing with an NC-1 Cortex which I sent in for an upgrade (New model A3 works great) - I could get a reading from the 1st IME in the chain but subsequent IME’s not so much. I never saw the Cortex ‘crash’ but having read about hte need to upgrade, I just requested it and it fixed the problem.

Good Luck trying to sort this out, Keep simplifying the test bed till you can isolate it to working reliably / not working .

Cheers Kb

Ours is an NC-1 that we sent in and has a ‘2’ written over the ‘1’. It sounds like you’re experience is similar in some ways to ours though. Maybe it is a problem with our cortex. At this point it definitely does not look like a code bug.

Well I’ve got some interesting results to share. I think I want to sleep on this and test tomorrow but here’s What I’ve found so far. All Tests conducted with Robot C 3.08 FW 9.12

First our team just got 4 new 393 motors and 4 new 393 based IME’s, my co-coach assembled the units (the screws are a bit sensitive) and I helped one of our students set up a test for two motor control and feedback.

Upon Testing we found that only one encoder seemed to be passing data back, upon some investigation it seemed that the IME in I2C slot one would pass data but not the IME in slot 2 We did a number of tests changing the motors back and forth. Also each motor and IME worked when it was the only IME plugged into the I2C port. (we tried 2 cortex units both nc2)

During these tests I did not change the code D/L’d to the Cortex I just powered off the cortex and swapped cables, turned it back on and ran tests again. We had code to run the motor and print the IME value every 1500 loops.

We checked the models of the Cortex we have four they are all NC2 although in all cases the ‘2’ is handwritten on the Cortex.

I decided to bring the 2 motors home for additional testing with the newer model A3 Cortex, I expected to see the devices work, But … surprisingly they did not behave the same. It appears that one of the IME’s is not fully functional. I labeled them Red and Blue. Blue seems to not produce data when in I2C Position 1 and gives data for a short while when in I2C position 2 . In the tests I did with the a3 version Cortex I spent a little more time and ensured the robotC setup matched the test configuration as far as number of I2C sensors in the chain.

As an added note the LED blink codes seemed to indicate the IME’s initialized and were in the expected place in the chain (see This Link) https://vexforum.com/t/vex-i2c-integrated-motor-encoder-device-spec/20600/1

I will try to repeat the Red Blue testing I did to ensure I see repeatable results, and I will try to bring my robot to school Thursday to see how my encoders work with the older NC2 Cortex.

At the end of the test I ran the test code using my robot IME’s and everything worked as expected.

So What may be happening is there could be one or more bad IME’s.

GregH did you ever test one IME at a time to see if you got good results with either one of the IMEs alone?

No, I haven’t tried that yet. I’ve been waiting to hear back from the Easy C folks in the hope that it was just a firmware bug or something that would make the problem go away :slight_smile:

We’ll try to do that today.

O.k. I’m interested in what you find out, It’s a good idea to collect as much relevant data on the problem as possible to isolate the root cause.

For example in my case right now I suspect there is something wrong with just one of the IME’s I have on hand I’ve tested the school IME motors with 3 of my own IME saddled motors I knew worked. The suspect IME seems to read 0 or 1 today for the most part wherever I try it in the chain. Actually I only ever tested 3 IME equipped motors at a time but 5 in all. I want to check out the other 2 our club received last week also I plan to retest all the encoders I can with the NC2 Cortex’s our school has.

Cheers Kb

I wanted to report that after testing the 4 new 393 IME’s 3 of the 4 work with both model Cortex’s. It looks like we got a bad IME. We will be in touch with Vex.

GregH You might want to do more testing to see if you can narrow down your problem. I didn’t test with Easy C as we don’t use it much anymore.

Good Luck - Kb