VAIC Status Report

VEX Robotics Competition Discussion
1

Hello VAIC Teams and the VEX Community

from Bob Mimlitch, Co-founder of VEX

It has been a long time since we announced the VEX AI Competition (VAIC) Pilot. The global pandemic has slowed down many teams’ abilities to participate in the Pilot. VEX has used our time to accelerate our hardware and software development for the next VAIC season. This status report is intended to inform the VEX community of the potential and future of VAIC. Many of the plans and information about the future of VAIC were never announced in detail, so I would like to take a minute and do that now.

We developed VAIC for several reasons. The purpose goes beyond wanting to make sure that VEX and RECF have the best robot competitions in the world. We heard from experienced VRC and VEX-U teams that they wanted bigger challenges.

VAIC Goals

  • An autonomous only challenge

  • More software engineering: object detection, GPS, communications, and strategy

  • More mechanical engineering: 3D printing, machining, new materials, and two robots

  • More electrical engineering: custom electronics, circuits and sensors are allowed

A little background first

VEX has been internally developing AI for use in competitions for some time now. Along the way we realized that without robot position information, the usefulness of object detection was limited, specifically for Teams programming an adaptive strategy. So we expanded our efforts to develop an indoor GPS specifically for VEX Competitions. We also realized that robots needed to coordinate their efforts, so we also added robot-to-robot communications to our growing list of development needs.

We realized along the way that the typical VAIC team might not have the time or experience to train an object detection model. For one reason it takes months of effort. For another reason, we use a custom dataset of hundreds of thousands of tagged images of VEX game and goal objects in a variety of locations, orientations, lighting conditions, etc. Training models takes significant experience with specialized tools. VEX, however, has one distinct advantage. We know the game a year in advance. We can develop, train, test and fine tune the object detection model and have it ready at Game Kickoff. This allows teams to make use of the information, without requiring every team to duplicate the effort year after year. Teams can still develop their own object detection methods if they desire. We have future plans to make this task easier.

VEX GPS Camera Sensor

The VAIC Pilot implementation of GPS used a separate Flir camera and processing on the Nvidia Nano, which is sent to the V5 Brain via USB. This method did not allow easy integration into VEXos and VEXcode. We plan to correct this with new hardware and software.

VEX is building a GPS Camera Sensor that captures VEX GPS Field Code images and calculates location. This sensor uses a monochrome camera to improve low-light performance and a global shutter to eliminate vibration from the images. An internal processor does all the position calculations in real time. This sensor has a V5 Smart Port to pass this data directly to the V5 Brain and VEXos, making the data easy to use.

The GPS Software has also improved significantly since the original release for the Nvidia Nano. The first GPS software improvement is that it can now find its position when viewing cluttered, fragmented scenes with only bits and pieces of the Field Code visible. Because this improvement helps the camera use codes that cover a larger angular span, the results are better triangulation and positional accuracy. Additional software improvements have sped up the time it takes to get an optical position location to 40 milliseconds. The second improvement in the software is made possible by the GPS Camera Sensor’s internal 6-axis Gyro and Accelerometer chip. A Kalman filter combines gyro and accelerometer data with optical position locations in order to arrive at position estimates every 5 milliseconds. This position estimate allows the sensor to continue to estimate position even if the GPS Camera Sensors view of the Field Code is completely blocked.

This sensor is expected to be available in May of 2021. The sensor kit will come with a new 12 foot long GPS Field Code strip that is more rugged and easier to install.

VEX AI Stereo Camera Sensor

The VAIC Pilot implementation of AI object detection used an Intel stereo depth camera and processing on an Nvidia Nano. The disadvantage of using the Nano is it’s bare PCB that needs a housing, the required fan for cooling, and the need for power from the V5 3-wire ports. Additionally, the nano can only communicate with the V5 Brain via USB, making software more complex than necessary. We plan to correct these issues with new hardware and software.

VEX is building a stand-alone AI Stereo Camera Sensor that integrates two cameras and more powerful AI image processing into a single device. This sensor has a V5 Smart Port to pass this data directly to the V5 Brain and VEXos, making the data easy to use. This sensor also has two USB host/device ports and built in 5.8 GHz WiFi.

The software for the AI Camera Sensor is undergoing significant improvements. Our goals for the 2021 Game will be increased object detection in low light and at a distance. In addition to object detection improvements, Kahlman filters will be used to reduce flickering in object detection. We will implement support for object detection issues. If a team is struggling to detect an object, they can send us images of their environment, and what their robot sees and we can create additional data and push an update. Getting this feedback will help us address all the environment scenarios.

Robot detection is currently not built into the VEX AI network because we lack sufficient data for VAIC robots and their design. This is largely due to the machining rules. To alleviate the issue of needing to know where all four robots are located, we will implement robot location wirelessly. VEXos will broadcast GPS location and receive all other robots locations. We will evaluate the usefulness of this once teams start using the new system.

This sensor is expected to be available in late Q3 of 2021.

Documentation in the Knowledge Base

When the new system is available we will have significantly more documentation to get teams up and running quickly. The topics are listed below.

  • Getting Started

  • VEX Field and Game Object coordinate system

  • VEXcode example code (C++ and Python)

  • Software API list and description

  • Using Object detection data

  • Driving to a position

  • Moving a manipulator to an object

  • Communication between robots

Final Thoughts

VEX and the RECF have put significant thought and effort into VAIC. We will continue to grow and evolve this program with the goal to keep it the best robot competitions in the world. We hope to see you all in person soon. Thank you all. Stay safe.

52 Likes
2

Will anyone be able to purchase the sensors, or will only ai teams be able to?

2 Likes
3

The sensors will we available to anyone. Usage in RECF competitions will be specified in the official rules.

25 Likes
5

This is all super amazing news. Individually purchasable especially sounds amazing so teams can try out some of the VAIC stuff to figure out if its something they have interest in.

30 Likes
6

Can you elaborate on the hardware that the AI Stereo Camera Sensor contains? What is the functionality of the USB ports? Is it possible to log into the sensor and make modifications as it is with the jetson currently?

Additionally, I appreciate the work that vex is putting into training the ML model for recognizing game objects. Are there any plans to release this model separately, or failing that, the dataset used to train the model? Both of these would be invaluable for teams planning to build their own object detection system.

16 Likes
7

Thank you for the updates Bob. Looking forward to the new GPS Field Code strip & Stereo Camera.

4 Likes
8

The AI Stereo Camera has a USB Host and Device port. Device is for connection to a computer. Host is for connection to the GPS for low latency location (as opposed to going from GPS to Brain to AI).

The hardware in the AI camera is the NXP i.MX 8M Plus.

Once the hardware is complete, we will look into ways to open up training. It is a little too early to discuss those plans, but we understand the need and want to make it possible.

16 Likes
9

Bob I just want to say that I really appreciate you making this post on the forum and answering our questions; the transparency about the future of VAIC is much appreciated.

This is great to hear! Releasing these tools will go a long way towards improving the educational ability of the program for advanced students.

This is really cool, and I look forward to seeing how this system will work.

This is interesting to hear. I take it that it would also be possible for a device other than the AI camera to read this data off the GPS (no encryption)? The increased modularity of the system (with the GPS and AI camera being separate modules) is very attractive, and it would be great if it were possible to read the GPS data directly from custom electronics

I have a few more questions:

  • Many advanced teams (the target audience for VAIC) are accustomed to using other environments for programming the V5 Brain, such as PROS. As far as I know, support for features useful in VAIC such as VexLink is not yet available to the developers of PROS. Are there any plans to make such functionality, as well as support for these new VAIC sensors, available in the 3rd party SDK in time for next season?

  • Will there be any form of DRM intended to prevent users that are not part of a registered VAIC team from using any of the hardware which you have mentioned?

  • Are you able to shed any light on what the registration fee for VAIC might look like this coming season? Specifically, will it be mandatory to purchase this hardware at the time of registration or will teams have more freedom to select their own hardware? I understand that this may still be under review

  • As you mentioned having VEXos broadcast the robot’s location to all other robots, will it be mandatory for every VAIC robot to make use of the GPS system so that their location can be known by all other robots?

Again Bob thank you for spending time to give us some increased transparency on the future of VAIC, it is much appreciated.

16 Likes
10

That’s something we will have to look at. The current implementation relied on running the scheduler used in VEXcode IIRC, so it’s not quite as simple as just opening up the API to the PROS team, it needs another phase of development and all the associated testing that comes with that.

15 Likes
11

No there will not be any DRM. Anyone can use the hardware outside of competitions.

RECF will have to answer this.

Yes. Use of GPS so this information is provided will be required.

12 Likes
12

I am extremely intersted in how the GDC plans on enforcing this. Do you have minimum ranges of field of view of the GPS that need to be unobstructed? Does it have to be mounted to the main chassis, or can it be on some appendage? How does that even apply in non-traditional robot types? It’s a very interesting problem.

I don’t want to give the wrong idea though; I am really interested and looking forward to what VAIC offers in the future.

12 Likes
13

Presumably it would be feasible to check at robot inspection that the GPS system is present and can see a sufficiently large portion of the barcode strip using some standard test (e.g., place the robot at the center of the field and rotate it through 360 degrees).

Perhaps they could also require that teams provide information about the robot’s maximum dimensions and the location of the GPS sensor within those dimensions, which also gets broadcast to all robots (and the accuracy of that data could also be verified at inspection), that way if someone builds a wallbot or something there’s some ability to tell where the whole robot is on the field vs. just the GPS sensor.

9 Likes
14

This sounds reasonable, as long as it’s objective and teams are actually able to do it on their own. However, it doesn’t quite take into account the fact that robots change throughout a match. What if I have a lift that occasionally obstructs it?

I do think that having this requirements will let us do much more interesting things with VAIC. However, I am also worried that it might unintentionally restrict engineering creativity in other ways, particularly on robot design. It’s all a question of both priorities and also how effectively this type of rule can be designed.

8 Likes
15

The tetherbot seen this year in Indiana comes to mind as a weird edge case for this. Maybe position of half the robot is reported, and the other half is “free to move about the country”. Perhaps future games will constrain robots to a maximum width/length as I seem to recall from Turning Point. Perhaps the first game will be more like “volleyball” or Star Struck where teams are constrained to half the field (I hope this is not what happens).

10 Likes
16

Hi Bob, since we are now about halfway through June, do you have an estimate as to when the new GPS kit will be available?

Thanks.

13 Likes
17

Hi Bob, We’re just about past halfway through Q3 and I’m wondering if you could give an update on the status of the AI Stereo Camera. Is it on track for release sometime next month? Has a price been decided upon?

Separately, are you able to shed any more light on the GPS location sharing system? Specifically, will it require any VAIC specific hardware for each field, or will it just be robot to robot wirelessly?

Thanks!

9 Likes
18

For the AI 3D WiFi Camera used for VAIC, delivery is still up in the air due to worldwide issues too many to list. Q3 is looking less likely and Q4 more likely.

The GPS location sharing requires no additional hardware on the robot or the field. Essentially the BLE radio in the AI 3D WiFi Camera will handle this robot-to-robot communications without any user intervention needed.

11 Likes
19

Hi Bob, We’re solidly into Q4. How’s the AI 3D WiFi Camera production looking?

7 Likes
20

Not sure what @Bob_Mimlitch_III will say, but it was announced a while back (Dan’s town hall a few weeks ago maybe?) that the season for AI would look a lot like last year as they would be available “before the end of the year”. Events will be extended for some time based on the electronics arriving and will culminate with a championship at IFI next summer. We could see the AI qualification date pushed weeks or months from the standard March 15th for VEX Worlds depending on when things are available and what the dates are for the championship event.

3 Likes