AI Vision on Visual Studio Code

Hello, as you know, VexCode Pro does not support new devices with the update. That’s why my team and I want to write our code with VEX plugin in Visual Studio Code application, but we couldn’t solve one thing; How do we set the AI Vision Sensor in this application?

Frequently Asked Questions
`-Why don’t we do it from the VexCode V5 block?
Since we use PID in our codes and write our own PID, we cannot switch to block code.

`How did you set the normal Vision Sensor before?
We never used it.

Robot config is not currently supported in the VEX Extension, so there is no way to configure an AI Vision Sensor using VS Code.

We are planning on releasing a separate Robot Config extension for Visual Studio Code. This extension will add support for the AI Vision Sensor and provide device configuration and code generation similar to VEXcode.

1 Like

Best short term solution is to use codev5.vex.com to configure colors for the aivision sensor. Copy the color descriptions and constructor into your VSCode project (and fix the silly names it uses), then you can follow example code for vision sensor as API is pretty much the same. An example.

// color descriptions
aivision::colordesc COL1(1,     193,      53,     104,   10.00,    0.20);
aivision::colordesc COL2(2,      37,     196,     168,   10.00,    0.20);
aivision::colordesc COL3(3,      35,     137,     248,   10.00,    0.20);

// create an aivision instance
aivision a1( PORT1, COL1, COL2, COL3 );

int main() {
    while(1) {
        // look for any color description
        a1.takeSnapshot( aivision::ALL_COLORS );
        
        if( a1.objectCount ) {
          Brain.Screen.printAt( 10, 30, "Object ID:%d", a1.objects[0].id );
        }
        else {
          Brain.Screen.printAt( 10, 30, "No objects     ");
        }

        // Allow other tasks to run
        this_thread::sleep_for(20);
    }
}
2 Likes

Thank you for all the answers, we will be looking forward to the configuration update for VSCode. :orange_heart::black_heart:

Where do you get these configuration values from VEXcode V5 so we can plug it into the configuration file for VEX Extension. I can set the color signature in VEXcode but the only values I see are: color ID, hue, saturation. What are these 6 values. I tried searching but could not find any documentation on this. I just know the first one is colour ID. Thank you.

It looks like AI Vision was recently updated, June 25, in VEX VS Code Extension v. 0.6.0 but not sure what’s supported and what the commands are.

  • Added Support for AI Vision Sensor:
    • VEXos updates
    • Device selection list specific for AI Vision Sensor. This allows device info to display information about a VEX Brain or Controller and an AI Vision sensor at the same time.

aivision firmware update is supported (ie. VEXos updates)
if an aivision and a brain are both connected, they show as separate devices.

In the section of auto generated code.

2 Likes

So, the number are
color id (1 through 7)
red value (8 bit, 0 - 255)
green value (8 bit, 0 - 255)
blue value (8 bit, 0 - 255)
delta hue, that is, the angle in degrees either side of the hue derived from the RGB value.
delta saturation, that is, the range of saturation either side of the saturation derived from the RGB value.

An example.
aivision::colordesc COL1(1, 222, 79, 85, 10.00, 0.20);

We can look at the range of colors selected on the color wheel (this is using an internal development tool)

Now change delta hue to 30 degrees
aivision::colordesc COL1(1, 222, 79, 85, 30.00, 0.20);

You can see a wider range of colors would be detected. A change of saturation would impact selection between outer and inner points of the color wheel.

For most VEX colors, red rings etc. the default values for hue and saturation range should be close.

3 Likes

Hello again, thank you very much for the new information. I have a small request if possible, is there any chance to see a code example that will turn the motor forward when it sees the Red Ring?

Thank you so much for the explanation but I’m still a little unclear as to where to get the configuration values. We use VEX VSCode Extension for our library but we’ve been using the vision utility in VEXCode V5 block to do the calibration as you suggested before. We followed the tutorial on how to set colours. However, the vision utility only show a swatch of the selected colour, hue range, and saturation range (see attached). Should we take the screen shot of the color swatch on the vision utility and use another program (e.g. photoshop) to find the 3 RGB value of that color? Is the delta hue and delta range the 2 number displayed on the Vision Utility (21 for hue and .38 for saturation from my screen shot)?

Or is there some other way to directly do it from VEX VSCode extension C++ now? Lastly, are there any new commands for AI Vision or basically the same as the vision sensor commands?
Thank you!

create a C++ project.
configure the aivision sensor, close the utility.
click on the disclosure “+” box for

#pragma region VEXcode Generated Robot Configuration

copy the constructors

Not yet.

It’s the same when using color detection.

1 Like

Thank you so much. I didn’t realize you can do that with VEXCode V5. This solved our problem. Thanks again.

On VEXCode V5, it uses aivision:

vex::aivision::colordesc RED(1, 7, 56, 188, 10, 0.2);

but in VS Code Extension, it does not seem to recognize vex::aivision:: only vex::vision::. We’re getting errors with aivision. The 2 setups does not seem interchangeable either as they have different parameters. Are we missing something?

make sure the project is using the latest SDK, if you started with an old project you may be on an old one.

image

1 Like

We weren’t able to figure out how to get to the project settings but we did manage to get aivision:: to work when we just set up the sensor and color without the rest of the sensor codes. There must be some sort other issue with the code. Thanks!

Do you know how many frames per second the AI sensor have?

The color detection algorithm runs at 30 fps.

1 Like

Thanks! So sounds like AI Vision is going to be a little slower to adjust heading than inertial sensor with typical 20ms wait between PID checks. Also, are there still plans to release game object recognition this year and if so, any idea of rough timelines for VS Extension?

1 Like

We just updated the AI vision sensor with a new firmware that was just released but now it seems it no longer works with VEX Code configuration.