Hi,
I am currently writing a replay autonomous (I am an experienced programmer, and I know what I am doing, the only reason we are doing a replay is because we don’t have time or a field to test it on). My replay algorithm works well, but I am unable to add the actual replay data to my program. Right now, I am storing them as follows:
//D1 is time, one reading every 20ms
//D2 is button/ch
//Autonomous
static const byte f_auton_ch [750][2] = {.....}
static const bool f_auton_btn [750][6] = {.....}
//Skills
static const byte f_prog_ch [3000][2] = {.....}
static const bool f_prog_btn [3000][6] = {.....}
But I end up with a
Out of static memory for variable allocation
error on the last declaration. Is there any way I can shrink these huge arrays down? I am already using bytes for the joysticks, and bools for the buttons. These are the bare minimum channels and buttons I need. (That is if my team can get an arm/claw working next week, if not, I can cut out 4 buttons). Any help is appreciated.
Some things you can look at:
-Booleans do not necessarily take 1 bit in memory. The compiler can decide if they should be packed or not. If they are not packed, you can pack them yourself.
-Do you need both programs stored at the same time?
Do you use all 15 seconds of auton and prog skills? If not, remove memory, even a few seconds from each helps
Button recording does not necessarily require a 3000 * 6 array of bool for all six buttons. You could do some data proccessing of button when recording and pack it all up. This way you could have a single array of 500 bytes for prog skills and each bit stores a press. If you are looking for an example of how to do this, this program uses something similar to do a large sieve of eratosthenes you can go off of.
You could store 8 booleans as a single byte. It would be more complicated to store/retrieve them, but it would save space.
As others have stated, there are better ways to do the data collection. When I wrote my replay program 6 years ago, I used a different method to get around the memory limitation, and could store close to 10 different autonomous routines. I don’t want to give away the method though, but you should be able to figure it out.
How are you planning for this to work? Are you planning to record a run on the field at the competition event, save the data in real time as the driver is moving, and then play it back a few minutes later all at the same event?
Do you intend the stored joystick/controller data to persist only while the Cortex remains powered?
Most people don’t do re-run that way. It can be more convenient to get the data out of robot by writing to the debugStream and capturing that over the wireless programming link. Often, the captured data is processed and used it to create an autonomous routine. The generated autonomous routine is then pasted into a competition template in the appropriate place, compiled, downloaded, and run.
Your way (which I think I’ve guessed correctly above) should work fine, but the stored values will only persist while the Cortex remains powered.
If that’s okay with you, then the only issue is to store the data compactly enough that you won’t run out of memory.
If you want help on compact storage methods, or on getting your recordings to persist by taking the data off-cortex, processing it, and getting it into either pre-loaded storage arrays or prebuilt code, let me know.
Actually, my recording program prints it all out to the debug stream (in this format, conveniently, so I can directly save the debug stream as a .h file). Then I just use an #include to include this data in my compiled program. This way, the data is saved as part of the program, survives power cycles, and is cortex-independent (so no file-system trickery). The algorithm I made just reads the array with the joystick values and emulates my driver control code so that it reads from the special globals I made. The issue comes with adding my array with data to the program.
Yes. I followed Eric’s advice and stored the booleans as bitmasks (bytes), so now I am able to store everything. But thanks for the help anyway!
Just out of curiosity, is there a way to further compress it? I thought of sparse arrays, but I didn’t know the best way to implement a Set datatype in ROBOTC.
Incidentally, my idea is exactly what you were talking about, except for the fact that I will only record once. I will go to the event, and record my autonomous and programming skills there. I am part of a private team, and we don’t have access to a field So you were perfectly correct!
Glad the bitmask was able to sort it out. In terms of ways to get even smaller
only storing skills OR autonomous at a time would save a lot
You could save about 1/3 of your used space if you were willing to only allow joystick values in multiples of 16. ( if you really wanted to try to be clever maybe 12)
You are going to process the data in a streming fashion, no random access necessary.
That way and given the nature of the data (you are hardly changing the controls every 20ms), you could easily implement a Run Lenght Encoding compression - prefix every sequence of the same values by the count. The “consumer filter” (during recording) would remember last value and don’t emit (store it to the output array) till a different value comes, so it would know and store the count (N) before the value. The “producer filter” (during playback) would just yield the same value N times before moving on.
If you learn that your values have noisy bursts that won’t compress (many cases of lengths of 1), you can go more fancy and have 2 length encodings, negative denoting sequence of changing values, positive for a repeated value, though that’s a little harder to compress in a streaming fashion.
Example:
The above works for generic data. For a domain specific data that you can mess with a little, imagine repurposing the lowest bit of data as a flag of count vs. value:
So you were perfectly correct!