Hi,
My team is new to VAIC, and we are wondering how to train the AI sensor to recognize game objects. We know that we can train for color codes, and we have made a basic recognition program using just color. I also know that the sensor can recognize last year’s game objects, and am wondering if/when it will be able to recognize this year’s. Is there any other platform we can use to help us with this?
Thanks!
Hi @Friday! I think you may be confusing three different things here. (VEX can have weird naming sometimes 
)
The first is the AI Vision Sensor. This is a V5 sensor legal for use in V5RC (and thus also in VURC, VAIC, etc.). The AI Vision sensor offers (to my knowledge) color blob detection as well as game element detection. I unfortunately do not know when VEX plans to add support for Push Back elements to the AI Vision sensor. You cannot train custom models using the AI Vision sensor (you technically can in VAIC, but that is another can of worms).
The second is the VEX AI System. This is purpose built for VEX AI (while the AI Vision sensor is broadly applicable to the V5) and uses an external coprocessor (an NVIDIA Jetson Nano) in conjunction with a depth camera (Intel Realsense d435) to run a custom VEX model. This VEX model (which is a YOLOv3n model if I recall correctly) is also pretrained, see this README on the High Stakes example repository. From what I see, it appears to have been trained on computed-generated imagery. You cannot train custom models for this system either (without quite a bit of work), but this model purportedly will have better accuracy for detecting game elements than the AI Vision sensor. The AI model for Push Back has also not yet been released. If I were to guess, I would assume they are releasing it around the same time as the AI Game Manual, which is expected in August.
The third is a custom model. The VAIC team I am on (3151A) trained and ran a completely custom model for High Stakes. We ran this on the same hardware that the VEX AI System uses (Jetson Nano + Realsense d435) but with our custom codebase and networking logic. This is definitely a lot harder than either of the above options, but can be quite rewarding. I can’t give super concrete numbers, but we observed a massive performance improvement (both speed and quality) over the VEX AI default model by training a custom model. The dataset we used spans about 600 images and is publicly available at Roboflow.
In order of accuracy, the AI Vision sensor is not super accurate (in my experience). The VEX AI System offers a relatively big jump. And if you are willing to jump through some hoops to run a custom model, that can be extremely rewarding. This is also the same order as effort needed; the AI Vision sensor requires very little setup, while getting the VEX AI system working might be difficult, while it takes a lot of effort to train a custom model.
In terms of costs, the AI Vision sensor is a much cheaper option than either of the latter two. In terms of 3151A’s costs, we used nearly the same hardware as the VEX AI System does (in fact, less), and spent less than 10 cents on GPU costs to train our model. I would estimate costs of the VEX AI system and custom model to thus be roughly similar.
Hope this helps!
Thanks for your explanation, it definitely helped our team a lot!