RobotC, EasyC speed tests

It has been assumed that as EasyC compiles to native code on the cortex, whereas RobotC uses opcodes being interpreted by a virtual machine, there is a significant advantage in terms of execution speed. I know speed tests have been done by others but I though I would run a few tests of my own to see what the advantage was.

Tests were done using RobotC V3.04 and EasyC V4.0.2.7. The tests involved setting a digital output on the cortex to a ‘1’, performing some action (ie. running some code) and then setting the digital output back to ‘0’. The time taken for the code to execute, that is the time the digital output was ‘1’, was measured using an oscilloscope. The code used was simple and translates between the two environments without much change, only the RobotC version will be shown as it’s easier to cut and paste here. The tests start trivial and gradually increase in complexity to finally end with a real world example from current competition code.

The first test was to just look at the time for the digital output alone to toggle.

task main()
{
    while(true)
        {
        SensorValue[dgtl1 ] = 1;
            
        SensorValue[dgtl1 ] = 0;
        }
}

RobotC time 3uS
EasyC time 1.5uS

Advantage EasyC (by a factor of 2)

Test 2 is a bit artificial and is the time taken to execute a for loop with 1000 itterations, this type of code is easy to optimize out but can be used as a poor mans delay in some situations.

task main()
{
    int x;
    
    while(true)
        {
        SensorValue[dgtl1 ] = 1;
        
        for(x=0;x<1000;x++)
            ;
            
        SensorValue[dgtl1 ] = 0;
        }
}

RobotC time 5.4mS
EasyC time 200uS

Advantage EasyC (by a factor of 27)

Test 3, now starting to get a bit more meaningful.

Set All 10 motors to a constant value of 0, ie. stop all motors.

task main()
{
    int x;
    
    while(true)
        {
        SensorValue[dgtl1 ] = 1;
        
        for(x=0;x<10;x++)
            motor[x] = 0;
            
        SensorValue[dgtl1 ] = 0;
        }
}

RobotC time 101uS
EasyC time 35uS

Advantage EasyC (by a factor of 3)

Test4, set all motors to the value of joystick channel 1

task main()
{
    int x;
    
    while(true)
        {
        SensorValue[dgtl1 ] = 1;
        
        for(x=0;x<10;x++)
            motor[x] = vexRT[Ch1];
            
        SensorValue[dgtl1 ] = 0;
        }
}

RobotC time 124uS
EasyC time 70uS

Advantage EasyC (by a factor of 1.8)

Test 5, send a string to the debug console.

task main()
{
    while(true)
        {
        SensorValue[dgtl1 ] = 1;

        writeDebugStreamLine("Hello");
            
        SensorValue[dgtl1 ] = 0;
        }
}

RobotC time 12uS
EasyC time 260uS

Advantage RobotC (by a factor of 21)

Test6, create a look up table used for motor control.

/*-----------------------------------------------------------------------------*/
/*                                                                             */
/*  Create a power based lut                                                   */
/*                                                                             */
/*-----------------------------------------------------------------------------*/


// Use a default of 20 if not already defined
#define MOTOR_LUT_FACTOR  20.0
#define MOTOR_LUT_OFFSET    10

// lookup table for non linear control
int   MotorLut[128];

void
MakeLut()
{
    int   i;
    float x;

    for(i=0;i<128;i++)
        {
        // check for valid power base
        if( MOTOR_LUT_FACTOR > 1 )
            {
            x = pow( MOTOR_LUT_FACTOR, (float)i / 127.0 );

            if(i >= (MOTOR_LUT_OFFSET/2))
               MotorLut* = (((x - 1.0) / (MOTOR_LUT_FACTOR - 1.0)) * (127-MOTOR_LUT_OFFSET)) + MOTOR_LUT_OFFSET;
            else
               MotorLut* = i * 2;
            }
        else
            {
            // Linear
            MotorLut* = i;
            }
        }
}

task main()
{
    while(true)
        {
        SensorValue[dgtl1 ] = 1;

        MakeLut();
                 
        SensorValue[dgtl1 ] = 0;
        }
}

RobotC time 14mS
EasyC time 31mS

Advantage RobotC** (by a factor of 2.2)

Make of these whatever you want, benchmarks are one way of looking at compiler speed but not the only way. They do suggest that for simple code execution EasyC is faster, however, as code complexity increases with the inclusion of math functions, RobotC has the advantage.*

Thank you so much for doing this research. I can’t express how awesome it is to have such good empirical data.

Seconded. This is really useful information to know, and even though realistically it shouldn’t affect most teams, it is very useful for hobbyists and college teams experimenting with electronics at the moment.

It does seem to depend on the type of instruction though. I would guess that anything that is just sending straight instructions to motors executes faster with EasyC, but anything that requires computation by the processor executes faster with RobotC.

I could be wrong, but I am recalling that last year, and as we added more code, we did have some serious issues with loop execution time using EasyC in Operator Mode. We basically had a big loop calling all the handler functions for the various subsystems, and it was getting slowed down to 20hz or so… this caused noticeable problems for the drivers who were getting laggy response when using the joystick to control the robot, and also affected the PID control loops I believe. Ironically, it could have just been the Print statement printing out how fast the loop was executing that actually was the major contributor to slow code; other print statements used last year as well did seem to significantly slow execution, as was demonstrated by jpearman.
This year, we are using RobotC, have a pretty big program, and so far, I haven’t gotten any complaints about lag. I can test this some time (I still have both year’s codes), and post back.
I don’t have access to an oscilloscope though… is there any other easy way to test execution speed? Maybe… using an audio recording program like audacity with a digital out from the cortex plugged into a mic port on a computer and observing the resulting waveform?

That might work. See also sites like soundcard_oscilloscope
Both languages have timers that you can use as an interval timer, then print the results afterwards. EasyC GetTimer block has resolution in mS (not uS per jpearman examples), so for small blocks of code, you may have to run them in a loop of 1000 times.
Welcome to the wonderful world of code profiling.

Any time you call a library, particularly an IO library like print,
there can be a large waste and variability of execution time,
so make sure that printing the results of the interval timer is not included in the interval being timed.

My favorite tool for this type of analysis is the Logic (or Logic16) from Saleae. It isn’t the cheapest hobby logic analyzer you can buy, but it has the feel of a quality tool (both hardware and software) and I find it very easy to use. Great for monitoring digital I/O ports to observe function entry/exit.

When I get some time, I’ll try to run these same tests on the VEXpro for another data point.

Cheers,

  • Dean

Nice post. Out of curiosity, in ROBOTC, was your Compiler Code Optimization set to “None” or “Full”?

Pretty sure I had optimization set to full as I did not want the array bounds checking turned on. These tests were done very quickly (although still accurately) in response to a comment in another thread. Both EasyC and RobotC have sufficient speed for most applications.

Just for comparison, I thought I’d run these same tests on the VEXpro. Not only are the time comparisons interesting, I thought folks might be interested in seeing the VEXpro version of the code to see how similar they actually are. These were built using TerkIDE and all the port access was via libwerk. The toolchain is gcc 4.3.4 with optimization level -O3.

The VEXpro version of this test:


int main()
{
	CQEGpioInt &gpio = CQEGpioInt::GetRef();

	gpio.SetDataDirection(0x0001);

	while(true)
	{
		gpio.SetDataBit(0);

		gpio.ResetDataBit(0);
	}
}

On the VEXpro, a full cycle (rising edge to rising edge) takes 838ns (approx 2x as fast as EasyC, and 4x ROBOTC)

The VEXpro version of this test:


int main()
{
	CQEGpioInt &gpio = CQEGpioInt::GetRef();
	volatile int x;

	gpio.SetDataDirection(0x0001);

	while(true)
	{
		gpio.SetDataBit(0);

		for(x=0;x<1000;x++)
			;

		gpio.ResetDataBit(0);
	}
}

The for loop takes about 75.5µS to execute on VEXpro, about 3x better than EasyC and 72x ROBOTC. I agree, though, that this is a pretty artificial test and doesn’t actually say much of substance.

Note: I had to change the optimization to -O0 (none) to get this to work. At -O3 it optimized-out the for loop even though x is volatile! (I’ll be looking into that later…)

The VEXpro version of this test:


int main()
{
	CQEGpioInt &gpio = CQEGpioInt::GetRef();
	CQEServo &servo = CQEServo::GetRef();
	CQEMotorUser &motor = CQEMotorUser::GetRef();
	int x;

	gpio.SetDataDirection(0x0001);

	while(true)
	{
		gpio.SetDataBit(0);

		for(x=0;x<12;x++)
			servo.SetCommand(x,0);
		for(x=0;x<4;x++)
			motor.SetPWM(x,0);

		gpio.ResetDataBit(0);
	}
}

The VEXpro has twelve 3-wire ports (1-12) and four 2-wire ports (13-16). Just to keep things similar, I set eight 3-wire ports (1-8) and two 2-wire ports (13 & 14).

VEXpro time was 12.25µs, about 3x faster than EasyC and 8x ROBOTC.
Zeroing all the 12 servo + 4 motor ports took about 21.85µs.

VEXpro can’t be joystick controlled yet. The closest comparison is probably opening a remote connection via Wifi and reading a value from a process on another machine. I’ll just skip this test - EasyC wins by default!

The VEXpro version of this test:


int test5()
{
	CQEGpioInt &gpio = CQEGpioInt::GetRef();

	gpio.SetDataDirection(0x0001);

	while(true)
	{
		gpio.SetDataBit(0);

		printf("Hello\n");

		gpio.ResetDataBit(0);
	}
}

This took 19µs most of the time, but 2ms sometimes. I think the 2ms time is when the console buffer fills up and/or is being sent over WiFi.

The VEXpro version of this code is identical (except for the different way of setting I/O Port 1), so I’ll omit it for brevity.

This took 6.9ms on VEXpro, about 2x faster than RobotC and 4x EasyC.

Cheers,

  • Dean

Thanks Dean, indeed very interesting.

I think I just measured the half cycle when the output was high, so 4x quicker than EasyC and 8x RobotC.

Good to know. All my other measurements were taken rising-edge to falling-edge so they should be a straight-up comparison.

I did a similar toggle test on the old PIC microcontroller using MPLAB and inline asm just to see how fast it could go. It easily exceeded the bandwidth of the digital I/O line. I seem to recall having to slow it down to around 1MHz in order to get enough voltage swing to hit the voltage thresholds when driving 1 TTL gate load.

From examining the FPGA verilog for the VEXpro I/O ports, it looks like they are limited to 781KHz (100MHz/128). All 16 lines are sampled synchronously at that rate, and are latched out at that rate as well.

The interrupt logic diffs two subsequent samples, so that makes the shortest safe interrupt pulse about 2.6µs. Pulses shorter than that may be ignored, depending on when they arrive relative to the sample clock.

I think the next test worth running in all these environments would be an interrupt latency test. I suspect the VEXpro might have a disadvantage there because interrupts are delivered to your program via Unix signals.

Cheers,

  • Dean