Use of AI in VRC

Programming Support
1

So I would consider myself to be a decent programmer. I have made a lot of unnecessary things for fun because I tend to have a lot of extra time during our robotics meetings. This year I was thinking about making a simulation, possibly in Unreal Engine or Unity, to train a neural network to make the highest scoring auton, and potentially react in real time during matches. Would this be allowed?

I know there aren’t actually any rules about not having a pre-programmed sequence run during driver skills or a match, but would a neural network be too far? Would it even be practical, using vision and ultrasonic sensors for the inputs?

4 Likes
2

I’ve thought about that multiple times.

That would be allowed, but probably not the best solution. It will be quite difficult to make a simulation in the first place. After that, there is the neural network, which will probably need hundreds of neurons.

The input and output system won’t be instantaneous but rather slow. Much slower than a human.

Training a human would have much less work, better reaction time, and would probably have a better outcome than an AI.

If you do somehow achieve this feat, then congratulations!

3 Likes
3

It absolutely is possible!
You will need to learn a bit of linear algebra if you want to apply it mathematically. However you can find a library on github and make a light neural network for your own use.

You can possibly add distance sensors around your robot (treated as lidar) with odometry wheels to estimate your robot’s pose on the field, and also use to drive around a robot during autonomous if they cross the auton line. In addition you can use neural networks to tune PID as well.

There are simplified ways to do machine learning without neural networks. A good method would be natural selection, which is you spawn many robots and slightly alter movement parameters. You keep the best 100 during a virtual run and delete the 900 that fail.

3 Likes
4

Yes! Another question about AI on the forum. Seems to be the only reason I stick around anymore. Implementing AI in VRC was my team’s greatest joy, and it showed me that even if it isn’t the most practical or successful route to take, it is definitely the fun one (and who knows, maybe in a hypothetical it could land you an internship because of the resume boost).

Long story short, my team implemented a machine learning model to control a grabbing mechanism in Tipping Point. This was then evolved to a fully autonomous, real time prediction driving model that was employed in Spin Up to block disks being shot by opponents in the autonomous period. And finally, the code was written into a C++ and python library to increase processing speed and reduce clutter. It is not under active development, but it could be a great reference to get you started! Reach out with any questions.

Copilot: A Machine Learning library for VRC

8 Likes
5

I have already made a few neural networks from scratch using C++, because there aren’t really any libraries lol.

Why would it be slower than a human? Is the brain not powerful enough to run a neural network? I wouldn’t train it on the brain, just run the finished product.

3 Likes
6

This is so amazing. I’ve never even heard about someone actually doing this, and I love your use case. I think your repo will be so useful to try and get started.

How did you train your AI? Did you train it on the brain in the field, or did you make a sim?

3 Likes
7

Appreciate the kind words! The data for the network was collected with standard data collection techniques for VRC. We logged data to a microSD that was in the brain, then took the .csv file we created and trained offline in python. This python program then exported a numerical weights .csv file that could be accessed in C++ on the brain.

The training protocol was a mix of real and artificial data. It included a lot of manual runs, driving the robot in front of a physical “shell” robot to get data that mimics opponents. This data was then buffed with a very simple simulation made in Turtle in Python to allow the program to adjust to a moving target that randomly performed one of four preprogrammed autonomous routines.

In an ideal world, I would have the time and programming expertise to have made this in Unreal (and actually, for a long time, that was the goal). Nevertheless, when you are panicking for data two days before the World Championships, a janky Turtle sim will suffice.

4 Likes
8

What did the original csv file on the brain contain? Was the a list of sensors? An initial training attempt?

Is it possible you could attach a screenshot of the sim? I made my own sim with pygame 2 years ago, but it didn’t work very well, and I want to see what level of complexity worked well enough for you.

Did having a robot that didn’t really resemble an actual bot impact the training? For example, what it learned was an opponent robot, vs just a disk or goal?

9

Also, @6210K , what sensors did you use?

If I use a mix of visual and ultrasonic, what would the neural network input be? Thousands of values for the pixels of the vision, then 2-3 values at the end for distance? Use two different networks on image and distance, and combine them?

1 Like
10

Good questions here. The exact .csv file we used to train the robot for Spin Up can be found in the Copilot github page under data/robot_data.csv. It was a combination of readings taken from motor encoders and laser distance sensors, slimmed down to the minimum number of inputs for an effective result.

I guess moderators can get mad at me for posting this huge chunk of code, but unfortunately, I can’t upload the python file itself, so here goes the simulation code, in its entirety, copied and pasted

import turtle
import math
import pandas
import numpy
import random

global log, count, pixels, values_list, movement_list, movement_count, finished
finished = False
log = True
count = 0
pixels = 0
movement_count = 0
movement_list = [0,0,0,0,0]
values_list = [0,0,9999,9999,9999,0]

screen = turtle.Screen()
screen.bgpic('field.gif')
screen.tracer(0,20)

opponent = turtle.Turtle(shape='square', visible=True)
opponent.color('red')
opponent.shapesize(1.2, 1.3, 1)
opponent.pu()

k = turtle.Turtle(shape='square', visible=True)
k.color('blue')
k.shapesize(1.4, 1.4, 1)
k.pu()
k.goto(-90, 60)
k.pd()

def run_auton(auton_num):
    global movement_list, finished
    finished = False
    if auton_num == 0:
        #Jugglenauts
        opponent.goto(-20, 90)
        opponent.seth(245)
        opponent.pd()
        movement_list = [2,2,2,2,2,2,2,2,2,2,0,0,0,0,0,-2,-2,-2,-2,-2,45,2,2,2,2,2,2,2,2,2,2,-45,0,0,0,0,0,-20,-2,-2,-2,-2,-2,2,2,2,2,2,20,0,0,0,0,0]
    elif auton_num == 1:
        #Speed of light
        opponent.goto(-20, 90)
        opponent.seth(180)
        opponent.pd()
        movement_list = [2,2,2,2,2,2,2,2,2,2,90,2,2,2,2,2,0,0,0,0,0,-2,-2,-2,-2,-2,135,2,2,2,2,-90,0,0,0,0,0,90,2,2,2,2,2-90,0,0,0,0,0]
    elif auton_num == 2:
        # Gears
        opponent.goto(-15,85)
        opponent.seth(270)
        opponent.pd()
        movement_list = [2,2,2,2,2,-150,0,0,0,0,0,0,135,2,2,2,2,2,2,-2,-2,-135,0,0,0,0,0,135,-2,-2,-2,-2,-2,-2,-2,-2,-2,-2,0,0,0,0,0]




def print_coords(x,y):
    x_coord = x
    y_coord = y
    print(x_coord, y_coord)

def full_forward():
    global log, count, pixels, values_list, movement_list, movement_count, finished
    LSD = 9999
    MSD = 9999
    BSD = 9999
    k.fd(2)
    screen.update()
    pixels += 2
    count += 20
    lsd_points = [(round(k.xcor() + 9*math.sqrt(2)*0.564705882352941 + a), round(k.ycor() + a)) for a in range(0,225)]
    msd_points = [(round(k.xcor() + a), round(k.ycor() + a)) for a in range(0,225)]
    bsd_points = [(round(k.xcor() + a), round(k.ycor() + 9*math.sqrt(2)*0.564705882352941 + a)) for a in range(0,225)]
    accepted_points = [(round(opponent.xcor() + 8*1.5625), round(opponent.ycor() + a)) for a in range(round(-8*1.5625), round(8*1.5625))]
    for x in range (0, 9):
        for c in [(round(opponent.xcor() + x*1.5625), round(opponent.ycor() + a)) for a in range(round(-8*1.5625), round(8*1.5625))]:
            accepted_points.append(c)
    for i in range (0, 9):
        for b in [(round(opponent.xcor() + a), round(opponent.ycor() + i*1.5625)) for a in range(round(-8*1.5625), round(8*1.5625))]:
            accepted_points.append(b)
    for x in accepted_points:
        if x in lsd_points:
            LSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-200, 200)
        if x in msd_points:
            MSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-200, 200)
            count = 0
        if x in bsd_points:
            BSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-200, 200)
    if log: 
        values_list[0] = 4
        values_list[1] = pixels * 0.030557749073644
        values_list[2] = LSD
        values_list[3] = MSD
        values_list[4] = BSD
        values_list[5] = count
    if movement_count == len(movement_list) - 1: 
        finished = True
        movement_count = 0
    else: 
        if abs(movement_list[movement_count]) > 2:
            opponent.rt(movement_list[movement_count])
        else: 
            opponent.fd(movement_list[movement_count])
    movement_count += 1
    
    if finished: 
        k.goto(-80,60)
        run_auton(random.randint(0,2))
    
    accepted_points.clear()
    print(values_list)
    
def full_backward():
    global log, count, pixels, values_list, movement_list, movement_count, finished    
    LSD = 9999
    MSD = 9999
    BSD = 9999
    k.back(2)
    screen.update()
    pixels -= 2
    count += 20
    lsd_points = [(round(k.xcor() + 9*math.sqrt(2)*0.564705882352941 + a), round(k.ycor() + a)) for a in range(0,225)]
    msd_points = [(round(k.xcor() + a), round(k.ycor() + a)) for a in range(0,225)]
    bsd_points = [(round(k.xcor() + a), round(k.ycor() + 9*math.sqrt(2)*0.564705882352941 + a)) for a in range(0,225)]
    accepted_points = [(round(opponent.xcor() + 8*1.5625), round(opponent.ycor() + a)) for a in range(round(-8*1.5625), round(8*1.5625))]
    for x in range (0, 9):
        for c in [(round(opponent.xcor() + x*1.5625), round(opponent.ycor() + a)) for a in range(round(-8*1.5625), round(8*1.5625))]:
            accepted_points.append(c)
    for i in range (0, 9):
        for b in [(round(opponent.xcor() + a), round(opponent.ycor() + i*1.5625)) for a in range(round(-8*1.5625), round(8*1.5625))]:
            accepted_points.append(b)
    for x in accepted_points:
        if x in lsd_points:
            LSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-200, 200)
        if x in msd_points:
            MSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-200, 200)
            count = 0
        if x in bsd_points:
            BSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-200, 200)
    if log: 
        values_list[0] = 0
        values_list[1] = pixels * 0.030557749073644
        values_list[2] = LSD
        values_list[3] = MSD
        values_list[4] = BSD
        values_list[5] = count
    if movement_count == len(movement_list) - 1: 
        finished = True
        movement_count = 0
    else: 
        if abs(movement_list[movement_count]) > 2:
            opponent.rt(movement_list[movement_count])
        else: 
            opponent.fd(movement_list[movement_count])
    movement_count += 1
    
    if finished: 
        k.goto(-80,60)
        run_auton(random.randint(0,2))
    
    accepted_points.clear()
    print(values_list)

def half_forward():
    global log, count, pixels, values_list, movement_list, movement_count, finished
    LSD = 9999
    MSD = 9999
    BSD = 9999
    k.fd(1)
    screen.update()
    pixels += 1
    count += 20
    lsd_points = [(round(k.xcor() + 9*math.sqrt(2)*0.564705882352941 + a), round(k.ycor() + a)) for a in range(0,225)]
    msd_points = [(round(k.xcor() + a), round(k.ycor() + a)) for a in range(0,225)]
    bsd_points = [(round(k.xcor() + a), round(k.ycor() + 9*math.sqrt(2)*0.564705882352941 + a)) for a in range(0,225)]
    accepted_points = [(round(opponent.xcor() + 8*1.5625), round(opponent.ycor() + a)) for a in range(round(-8*1.5625), round(8*1.5625))]
    for x in range (0, 9):
        for c in [(round(opponent.xcor() + x*1.5625), round(opponent.ycor() + a)) for a in range(round(-8*1.5625), round(8*1.5625))]:
            accepted_points.append(c)
    for i in range (0, 9):
        for b in [(round(opponent.xcor() + a), round(opponent.ycor() + i*1.5625)) for a in range(round(-8*1.5625), round(8*1.5625))]:
            accepted_points.append(b)
    for x in accepted_points:
        if x in lsd_points:
            LSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-200, 200)
        if x in msd_points:
            MSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-200, 200)
            count = 0
        if x in bsd_points:
            BSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-200, 200)
    if log: 
        values_list[0] = 3
        values_list[1] = pixels * 0.030557749073644
        values_list[2] = LSD
        values_list[3] = MSD
        values_list[4] = BSD
        values_list[5] = count
    if movement_count == len(movement_list) - 1: 
        finished = True
        movement_count = 0
    else: 
        if abs(movement_list[movement_count]) > 2:
            opponent.rt(movement_list[movement_count])
        else: 
            opponent.fd(movement_list[movement_count])
    movement_count += 1
    
    if finished: 
        k.goto(-80,60)
        run_auton(random.randint(0,2))
    
    accepted_points.clear()
    print(values_list)
    
def half_back():
    global log, count, pixels, values_list, movement_list, movement_count, finished
    LSD = 9999
    MSD = 9999
    BSD = 9999
    k.back(1)
    screen.update()
    pixels -= 1
    count += 20
    lsd_points = [(round(k.xcor() + 9*math.sqrt(2)*0.564705882352941 + a), round(k.ycor() + a)) for a in range(0,225)]
    msd_points = [(round(k.xcor() + a), round(k.ycor() + a)) for a in range(0,225)]
    bsd_points = [(round(k.xcor() + a), round(k.ycor() + 9*math.sqrt(2)*0.564705882352941 + a)) for a in range(0,225)]
    accepted_points = [(round(opponent.xcor() + 8*1.5625), round(opponent.ycor() + a)) for a in range(round(-8*1.5625), round(8*1.5625))]
    for x in range (0, 9):
        for c in [(round(opponent.xcor() + x*1.5625), round(opponent.ycor() + a)) for a in range(round(-8*1.5625), round(8*1.5625))]:
            accepted_points.append(c)
    for i in range (0, 9):
        for b in [(round(opponent.xcor() + a), round(opponent.ycor() + i*1.5625)) for a in range(round(-8*1.5625), round(8*1.5625))]:
            accepted_points.append(b)
    for x in accepted_points:
        if x in lsd_points:
            LSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-100, 100)
        if x in msd_points:
            MSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-100, 100)
            count = 0
        if x in bsd_points:
            BSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-100, 100)
    if log: 
        values_list[0] = 1
        values_list[1] = pixels * 0.030557749073644
        values_list[2] = LSD
        values_list[3] = MSD
        values_list[4] = BSD
        values_list[5] = count
    if movement_count == len(movement_list) - 1: 
        finished = True
        movement_count = 0
    else: 
        if abs(movement_list[movement_count]) > 2:
            opponent.rt(movement_list[movement_count])
        else: 
            opponent.fd(movement_list[movement_count])
    movement_count += 1
    
    if finished: 
        k.goto(-80,60)
        run_auton(random.randint(0,2))
    
    accepted_points.clear()
    print(values_list)

def stay():
    global log, count, pixels, values_list, movement_list, movement_count, finished
    LSD = 9999
    MSD = 9999
    BSD = 9999
    screen.update()
    count += 20
    lsd_points = [(round(k.xcor() + 9*math.sqrt(2)*0.564705882352941 + a), round(k.ycor() + a)) for a in range(0,225)]
    msd_points = [(round(k.xcor() + a), round(k.ycor() + a)) for a in range(0,225)]
    bsd_points = [(round(k.xcor() + a), round(k.ycor() + 9*math.sqrt(2)*0.564705882352941 + a)) for a in range(0,225)]
    accepted_points = [(round(opponent.xcor() + 8*1.5625), round(opponent.ycor() + a)) for a in range(round(-8*1.5625), round(8*1.5625))]
    for x in range (0, 9):
        for c in [(round(opponent.xcor() + x*1.5625), round(opponent.ycor() + a)) for a in range(round(-8*1.5625), round(8*1.5625))]:
            accepted_points.append(c)
    for i in range (0, 9):
        for b in [(round(opponent.xcor() + a), round(opponent.ycor() + i*1.5625)) for a in range(round(-8*1.5625), round(8*1.5625))]:
            accepted_points.append(b)
    for x in accepted_points:
        if x in lsd_points:
            LSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-100, 100)
        if x in msd_points:
            MSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-100, 100)
            count = 0
        if x in bsd_points:
            BSD = round(16.255999479808015 * k.distance(opponent)) + random.randint(-100, 100)
    if log: 
        values_list[0] = 2
        values_list[1] = pixels * 0.030557749073644
        values_list[2] = LSD
        values_list[3] = MSD
        values_list[4] = BSD
        values_list[5] = count
    if movement_count == len(movement_list) - 1: 
        finished = True
        movement_count = 0
    else: 
        if abs(movement_list[movement_count]) > 2:
            opponent.rt(movement_list[movement_count])
        else: 
            opponent.fd(movement_list[movement_count])
    movement_count += 1
    
    if finished: 
        k.goto(-80,60)
        run_auton(random.randint(0,2))
    
    accepted_points.clear()
    print(values_list)

def auton_block(x,y):
    run_auton(random.randint(0,2))
    k.fd(10)
    k.rt(45)
    screen.update()
    screen.onkeypress(full_forward, 'w')
    screen.onkeypress(full_backward, 's')
    screen.onkeypress(half_forward, 'i')
    screen.onkeypress(half_back, 'k')
    screen.onkeypress(stay, ' ')
    screen.listen()



screen.update()
screen.onclick(auton_block)
screen.mainloop()

As you can see, a janky mess. I am struggling to even recall what most of it means. I guess the lesson is use comments, friends.

And lastly, maybe it did impact the training not having a real robot to compare to, but this whole thing was put together in the 3-ish weeks between our State competition and Worlds, so all we cared about was getting it working. The whole point was to try to position the robot directly in front of an opponent, so the shell sufficed. I’d say it had about a 60% success rate at Worlds, which is way better than 0, and we didn’t lose any offensive firepower because our robot was defensively built for gameplay and had a very low scoring ability. Overall, it just made sense for us, even if it wasn’t the best it could be.

5 Likes
11

Sorry, but could you define the values in the robot_data.csv? I don’t fully understand how they correspond to the inputs for the network. The values are:
Speed
Pos
LSD
BSD
MSD
Time_Since_MSD (This seems self-explanatory, but the values are very…robotic)

1 Like
12

Yeah, forgot to address that. The sensor setup was such that we had distance sensors evenly spaced on the right side of our robot, such that when it drove forward it could detect what was next to it. The values you see are the raw values of those sensors being logged every 20 ms during trial runs.

These inputs can really be anything you are trying to find a statistical connection between. We were just trying to find a connection between these values and the position and speed of the robot so it could learn to keep up with opponent robots and their movement.

1 Like
13

Only on the right side of the robot? Why not on the left side? Or a few on the front?

And thank you, that clears it up.

14

The robot drove forward along the diagonal auton line in Spin Up, and never had to be anywhere except right on the line to maximize size. Also, the auton only ran on one side of the field, so we only needed one side of sensors.

2 Likes
15

I haven’t done a lot of research into this, but what type of training does vex AI use? I would guess reinforcement, but I couldn’t find anything in a quick Google search.

16

I feel like this is an amazing idea, and if you get out working, I’d love to hear more!

My issue would be all code needs to be running on the brain, and I’m not sure it can run a full AI system, especially with UE5 or Unity.

But please do update us on your progress, even if it doesn’t get to competitions!

Good luck!

17

I’m going to only be running the trained network on the brain. The sim, and the backpropagation will be done on an actual computer. I don’t think forward prop is going to be too intensive for the brain.

1 Like
18

I am a coach during the year, and over the summer I write lots of open source software to give away to everyone. I also use the software to teach engineering concepts with my students.
I’m not sure if it would help you, but I built a CAD tool for modeling and simulating Vex robots and the High Stakes game. It uses the BowlerStudio runtime and the MuJoCo physics engine. I taught an engineering and simulation class last year and made this platform the final lesson, since the whole class was on our Vex team. It is free, open source and works Windows/Mac/Linux. It can be used by you to create the game and the AI stack, instead of having to start from scratch in Unreal or Unity.
BowlerStudio can be installed here: https://commonwealthrobotics.com/
A sample “robot” can be loaded from here: GitHub - madhephaestus/VexVitaminsRobot: Project VexVitaminsRobot
And the game and field is defined here: GitHub - madhephaestus/VexHighStakes2024: Field and elements for Vex High Stakes
The robot is the physics model of a robot, without any of the connecting pieces, so it is not a build-able design. The goal was to have each team design their robot in simulation and test it with the multi-contact physics engine before building. You may be able to use the highly accurate physics modeling to make your AI model correct according to physics.

Screenshot from 2024-09-04 15-58-49
Screenshot from 2024-09-04 15-58-491920×1306 177 KB

9 Likes
19

This looks awesome, thank you. Is it programmable? Am I able to add a C++ neural network to the back end to control the robot?

1 Like
20

Yeah it is programmable! The programming environment supports JVM langauges (Java, Groovy, Clojure) and all of the example code is in Groovy. It has full Eclipse support for the code editing which provides intelasense method prediction and step-through debugging. Since its all in the common JVM, the step through debugger works WITH the physics engine allowing you to debug real-time algorithms.

Since you had already planned to have the training and simulation to be a totally different program from the runtime, it may still be a useful tool for you!

3 Likes