How To Controll Brain through Computer

VEX V5 Technical Support OFFICIAL Tech Support
1

Hi,
Is there any way that my team can controll the brain through a PC? We are running a machine learning script that decides which way to move.

2

Yes try using the VEXcode Text coding studio.

VEXcode Studio

3

Sounds interesting… could you elaborate on what exactly you want to do?

4

We want to stream video from a phone to a computer so it can detect where the objects are. From that we will testing different machine learning approaches and coding which strategy is the best. The computer will act as a controller that takes outputs from code and inputs to the robot brain.

5

Thank you! We will try this.

6

You might need to use PROS if you want to setup a CLI to set motor speeds/send commands. I don’t think VEXcode has this functionality yet.

7

Nope (as long as you want a wireless solution, which I benevolently assume here).

So let’s stay with wireless options:
VexCode does have a channel from the robot to the PC - printf() in the program will get transferred over the wireless link to the controller, which will pass it through the USB port to the terminal inside VexCode. I am not aware of a backward channel, neither of an API to plug your own stream processing to the console I/O.

PROS is even worse in this regards (so far, to my best knowledge, will verify) - the wireless console doesn’t work under PROS yet (there is open Issue #94 and proposed patch on a branch, but not in release AFAIK.

The most promising approach currently available (somewhat) is the new vex::vex_link API that is supposed to allow bidirectional robot-ro-robot (read: brain-to-brain) communication. You’d keep a second brain wired to the controlling computer and provide it with the data to forward over the VEXnet. This brain could use various wired means to talk to the computer - if nothing else, you can at least use one of the V5 ports in the serial port mode, hook it up to USB-RS485 transceiver and feed it that way, but the vex_link suggests it might provide another avenue.
(Unfortunately, I can’t find much documentation on vex_link so far, besides the vex_vexlink.h header in VexCode…)

8

in user program you can just use std::cin (or scanf) , you can send data from the VEXcode terminal or any other program that opens the user serial channel. This works for direct USB or wireless via the controller.

4 Likes
9

Oh, cool then.
Need to try that!

10

actually, that was incorrect, wireless io only works from the VEXcode terminal. The issue is that the controller only has a single serial channel, so we have to wrap all the stdio messages in our admin protocol. direct USB has two channels available, so we can dedicate one to user code.

3 Likes
11

You want to be using PROS. Since it integrates directly with atom, which has deep learning support, it should have a package downloadable for machine learning integration with PROS. If that will not work, then we can collaborate to create an adapter for PROS. Cheers,

Yug

12

I will definitely have to do some more research thank you all!

13

Atom has noting to do with that. Atom is just an IDE. I use PROS without Atom, while you can even use Atom to develop for the VexCode platform (w/o upload support)

1 Like
14

@jpearman is the Joystick USB channel mux protocol available somewhere? That might be a viable approach for OP, implement enough mux into their ML interface to be able to stream data wirelessly to the brain.

1 Like
15

No, it’s not publicly available.

16

Edit: Updated message format below…
So it turned out wireless console I/O through the joystick/USB is relatively simple. With no complicated setup, all you’d need to implement on your ML side is wrapping your data in a simple message structure and possible unwrapping the data from the reply message.

What programming environment do you use for the ML side? How flexible are you with the interfaces?
I tried a simple C program on Linux, accessing the Joystick connected through USB (visible as /dev/ttyACM0 on my Linux box). While a program is running on the brain, you can reach its console exchanging 0x27 messages with the following structure:

PC->Brain:
c9 36 b8 47 | 56 | 27 | 04 | 01 | 02 | 33 34 | 3b 23

c9 36 b8 47: is the vex synchronizing sequence, always fixed
56: means extended command
27: is the user IO command
04: is the len of the extended message (2B + payload)
01: would be for using download channel??
02: sending 2 bytes
33 34: my payload ( ‘3’, ‘4’)
3b 23: CRC16 of the whole message

Brain->PC:
aa 55 | 56 | 1f | 27 | 76 01 | 52 65 63 65 69 76 65 64 20 27 33 27 0a … | a0 63
aa 55: Start of reply frame
56: extended message
1f: length of the following data, 1 or 2 bytes. 1 byte up to 0x7f, if more data is coming (up to 0x7fff), the first byte is the MSB of length (ORed with 0x80), second byte is the LSB
27: reply to command 0x27 (user I/O)
76 01: Don’t know yet
52 65…: “Received ‘3’\n” … - console output
a0 63: CRC

Now, that would work under VexCode I presume. Under PROS, things are little more complex, since PROS uses another layer of console multiplexing, separating stdout with stderr. You can either try implementing it on your side too, or you can disable it using pros serctl function, see: Filesystem — PROS for V5 3.8.0 documentation

The exchange above was using a simple PROS program that did serctl(SERCTL_DISABLE_COBS), then it run a simple loop of:

while (true) {
    int c = getchar();
    if (c > 0) printf("Received '%c'\n", c);
    pros::delay(100);
}
1 Like
17

You might to try out RMS studio. I remember that RM had an interesting set up with bidirectional serial communication at their worlds booth last year. @Rick_TYler or @John_TYler might be able to help here.

19

The program we were running on the computer at Worlds was a dedicated one-off for that demonstration and we have not incorporated freeform bidirectional communications into RMS. Sorry D:

20

Thank you for your very detailed reply. So this:

FILE* stderr = fopen(“serr”, “w”); fputs(“Example text”, stderr); fclose(usd_file_write);

would this only disable COMBS? Or could we write commands instead of the example text that the controller can take?

21

I am not sure I understand your question, so I’ll try to provide broader answer.
For your solution, there are multiple components.

One of them is the code running on the robot. on the V5 brain. That piece is receiving the commands from the remote computer and drives the motors. It could be a smart, complicated system that understands high level commands, but for the sake of simplicity, let’s make it dumb, only directly commanding 2 motors based on direct values (akin to teleop, except that the remote is a computer, not a human with a joystick). Let’s also think of simple text protocol: a pair of integers and a newline for every command. Your V5 program would look like:

while (true) {
    int left, right;
    int values = scanf("%d %d\n" &l, &r);
    if (values == 2) {
        leftMotor.setSpeed(left);
        rightMotor.setSpeed(right);
    }
    sleep(10_milliseconds);
}

That is, the code is just reading pairs of numbers off stdin and applies them to a pair of motors.

Another component would be your camera and ML setup, that’s beyond the scope of this question, but let’s say it generates the 2 motor speed numbers on the remote computer and passes them somehow to the third component:

The third component is a function in your remote computer ML system that is responsible for sending those two numbers (the command) to the robot. Depending on your OS and programming language, it would look differently, so let me show how a minimalist approach could look like with C and Linux:

void sendCommand(int l, int r) {
    char textBuf[64];
    char msgBuf[128] = "\xc9\x36\xb8\x47\x56\x27";

    static int serial_fd = open_serial("/dev/ttyACM0", 115200); // open once and keep around FD for the serial port

    int textLen = snprintf(textBuf, sizeof(textBuf), "%d %d\n", l, r);
    msgBuf[7] = 2 + textLen;
    msgBuf[8] = 1;
    msgBuf[9] = textLen;
    memcpy(msgBuf+10, textBuf, textLen);
    uint16_t crc = doCrc(msgBuf, textLen+10);
    msgBuf[10+textLen] = crc >> 8;
    msgBuf[11+textLen] = crc & 0xff;

    write(serial_fd, msgBuf, textLen+12);
    // read and perhaps validate the reply message from the brain
    // but you'd better set the serial as non-blocking... 
    read(serial_fd, msgBuf, sizeof(msgBuf));
}
1 Like