Integrated Motor Encoder Issues

VEX V5 Technical Support UNOFFICIAL Tech Support
#1

We started using IMEs recently and we’ve been having a host of strange seemingly meaningless issues (we’re using PROS).

I think it’s best to describe what happened in time order:
We initialized the encoders and everything seemed to be working fine (we were getting the correct values just as we expected based on wheel turns in the LCD display). However, we have a method to record autonomous and programming skills routines (for which we want to use the encoder), and simply adding the single line “imeInitializeAll()” caused this routine to move random motors on our robot. This is very odd, since for debugging purposes, we had this routine NOT use any encoder values, which means that the act of initializing the encoders itself caused the strange motion (and not the fact that encoder values were not zero). Curiously, driver control worked normally, which led us to think that perhaps memory overflow or some sort of memory corruption was occurring (screwing up our autonomous recorder, which uses around 6 kb of memory). Later, the cortex randomly started to crash just after we finished saving a file (this was working perfectly before the IMEs were added) and gave a blank error code over serial. This consistently occurred every time we tried to save an autonomous file. About an hour later, we tried testing once more, except this time, “imeInitializeAll” returned 0 every time and the lights on the encoders remained yellow, even though we changed nothing about the connections. Based on previous posts, we considered the possibility that built up static charge could have affected our IME performance, but we can’t seem to understand how that would cause all these symptoms. Any help is welcome :confused:

#2

Are you writing files to flash ? When do you do this?

When do you call ImeInitialize All() ? Did you read the notes about it not being thread safe ?

You are probably going to have to post your code, or at least a small example that show the behavior you see.

#3

Hi jpearman, I’m a member of the same team.

The imeInitializeAll() function is called in the initialize() function that the PROS kernel calls automatically. This is the only time this function is called, so the fact that the function is not thread-safe should not be of concern here.

In the routine in which the Cortex crashes, we are indeed writing files to flash, but at that point, all the file writing has been completed. The Cortex most likely crashes when closing the file ( fclose() ).

Do you know what could be causing this issue?

#4

Scratch that, it closes the file successfully (according to our print statements).

It crashes when exiting the function.

#5

Just send me a link (or PM) me the code, probably the quickest way to get this sorted. Using the flash is tricky, interrupts can be disabled for some considerable time and this can disrupt the communications between master and user processor. That may not be your problem here but without any more information it’s the most likely suspect.

#6

We figured out a little more about the issue. As we save the file, we printed out what should be saving over the serial port, and the numbers all look fine. However, when we load the file from the PROS IDE and view it in a hex editor, the values look normal until about 1/4 of the way through, at which point the file appears to have random values.

#7

Could static discharge be causing these file writing errors to occur? After a day of not touching or using the robot, the file saving worked, and after that one time, it did not work again.

#8

I doubt static is the issue.

There are a number of things that may be going on, but before we try and debug via the forum let me try the code you sent earlier and see what behavior I see. Unfortunately I cannot do that until late tonight or tomorrow morning. Assuming there is not a bug in PROS, I’m sure we can come up with a fix or workaround that allows your code to work.

#9

Just an update.

The code does not crash on me, however, file write in PROS does appear to be broken. Anytime data is 0xFF (or -1, same thing) the write command does not write enough bytes to the file.

An example.

void operatorControl() {
    char    data[8] = {0,1,2,3,0xFF,0,0,0};
    int     s;

    FILE *fp = fopen("james", "w");

    s = fwrite( data, 1, 8, fp );
    fclose(fp);

    printf("Wrote %d bytes\r\n", s );

    while (1) {
        delay(20);
    }
}

should write 8 bytes, only 4 seem to be written, file size is set at 5 which is also wrong.

Same thing in C

#include <stdio.h>

int
main( int argc, char *argv] )
{
    char    buf[8] = {0,1,2,3, 0xff, 5,6,7};
    FILE    *fp;
    int      l;
    
    fp = fopen("james", "w");
    
    l = fwrite( buf, 1, 8, fp );
    fclose(fp);
    printf("wrote %d bytes\n", l );
}

Works correctly.

This probably accounts for your files being truncated.

So I’m not yet sure what other problems you may have but the PROS file system seems to be broken. We need some inout from Jabibi.

When the cortex crashes, what happens? Any chance you have filled the file system? In other words, does this happen after new code has been downloaded and all files deleted?

Edit:
A workaround for write is probably to use fputc in a loop. This seems to work.

void operatorControl() {
    char    data[8] = {0,1,2,3,0xFF,0,0,0};
    int     s;
    int     i;

    FILE *fp = fopen("james", "w");

    for(i=0;i<8;i++)
        fputc( data*, fp );
//    s = fwrite( data, 1, 8, fp );
    fclose(fp);

    while (1) {
        delay(20);
    }
}

The way your code is written it may be easier to just use 8 fputc statements with the variables that need to be saved.*

#10

Thank you for the prompt reply jpearman. I’ll try using fputc to put the characters one by one into the file, but this does not explain why this issue does not occur when imeInitializeAll() is not called in the initialize() function. I’ll be able to test out fputc vs. fwrite in the afternoon today.

#11

It does not explain that.

I need to know more about the condition that causes the cortex to crash, what happens, red flashing led? loss of control? Smoke?

Does the code finish the loop writing data to the file? If so, how much further does the code run. Does the crash occur if the flash is completely erased (ie. download new code) or have several files been written, I’m wondering of the file system area of flash (which is about 250K) has been filled, I don’t know what happens in that situation. How old is this cortex? There’s a small chance that a bad flash sector could cause this behavior, not impossible with an old cortex.

Last night I only had two encoders installed, I will add two more and see of that makes any difference.

#12

The crash consistently happens when the saveAuton() function is about to exit. The only indication of this that we have noticed is that the code restarts from the beginning and a serial message is printed saying that the Cortex has crashed. It finishes writing and has closed the file at that point. The file space has certainly not been filled at that point; when the issue occurred there were no other files written to the Cortex.

#13

Hmm, interesting, sounds more like a watchdog reset than a crash. I tried this morning with four encoders but no difference, code does not crash for me. Any chance you can try on another cortex? How old is the one you are using? Also, how long are the encoder 4 wire cables, can you try with just one plugged in? What version of master code are you running, I was testing on 4.23 here.

#14

Strangely, the issue with the crashing is no longer occurring after a day of not touching the robot. However, we still are having the file corruption issue. To answer your questions, we can test on another cortex, this cortex is about 2 years old, we have about 10 inches of 4 wire cable between each encoder, and we’re running 4.23. Hopefully, we’ll be able to try with just one plugged in soon (our drivers are practicing now :slight_smile: )

#15

Pretty sure the file corruption is caused by the write bug I explained below, just use fputc and that should go away. It’s possible the crash is caused by something else completely, certainly static can cause the IMEs to lockup the cortex but if that were happening there’s no reason why it would occur just as the file was being written.

#16

Ok, we will try fputc() as soon as possible (we’ve written the code for it, just need the robot to work on). However, during operator control the robot randomly disconnected from the joystick and motors continued to move in the direction they were moving in before, causing the robot to break. Aren’t motors supposed to stop if the cortex disconnects from the joystick? We believe that the cortex most likely crashed or froze, although we do not have serial output to confirm this (joystick was plugged into the wall).

#17

Runaway motors was/is a common problem that we had during the sack attack season. There is a bug in the master firmware that means if the user cpu stops communicating with it the motors are left running at the last commanded speed. The IMEs are susceptible to static and can cause a user cpu crash, we spent a lot of time just before worlds in 2013 improving the reliability of IME communication for ROBOTC and added a watchdog timer that would at least reset the user cpu if the code crashed. I don’t know what the PROS team has implemented, the watchdog timer in ConVEX is optional and up to the user to turn on as needed (I don’t particularly like watchdog timers, they just are a band aid to some other systemic problem usually).

So it may be that your file write problems and cortex crashes are unrelated.

One other possible issue is that you have a lot of communication in the code using printf statements, perhaps turn that of in the operator control loop, it may help (leave all the other printfs in the file save etc.).

#18

Thank you so much jpearman!

The file writing issues have been “solved” (worked around) by using fputc() instead of fwrite().

We are adding a static discharger of sorts by screwing the end of a wire connected to the Cortex down to a piece of metal attached to the robot in order to dissipate static electricity. Do you think this will alleviate the issue or should we try something else?

#19

We cannot say for certain if you have a static issue, however, search back through the forum and read about what was said in the past. TL;DR version is that there is no good solution that we really found and nothing much that the engineers at VEX could suggest. There’s some background in posts #1 and #2 in this thread.
https://vexforum.com/t/ime-technical-description-and-software-workarounds/23218/1&highlight=technical

During the Sack Attack season there were many robots with polycarbonate intakes, this combined with the sacks probably made things worse. There is also (an unproven) theory that mecanum wheels cause more problems and that IMEs on wheels are more likely to suffer than those on an arm mechanism, all just anecdotal but that’s also my own experience.

So what is the current situation? Is the cortex still crashing? Does it still only happen after the file write or can it be random during driving?

#20

Just for your information, we’re using omni wheels, not mecanum wheels. Also, the crashing doesn’t appear to be occurring anymore, I think the error was in fwrite as you identified. The main issue we’re having now is that the IMEs don’t always initialize during the init() function. What we’ve implemented right now is a delayed counter that initializes the encoders continuously until all four of them are initialized (something that apparently worked for another team in our club), but it’s not exactly working and it seems to be random whether or not the encoders initialize.