Filtering of sensor data?

  1. 3 months ago

    biglesliep

    Mar 21 Menlo Park, CA 1666

    I have seen a number of forum posts over time (such as this info-packed one) mention applying a "low pass" filter or a Kalman filter to account for noisy-values of sensor output. I get the point -- it doesn't help a while-sensorData-loop to have wacky numbers randomly appearing.

    My questions are:

    • how much improvement are we talking about compared to no filter?
    • do a lot of teams do this, or just a few with really sophisticated programmers?
    • how do you even know if you need to filter?
    • what sensors do you do filtering with? (pot, ultrasonic, shaft encoders, gyro...)
    • how hard is it to combine filtering with other algorithms, like PID?

    Thanks all --

  2. Vex 9185

    Mar 21 Bellefontaine, Ohio

    I have some limited experience with filters.

    My first experience was back in Nothing But Net, when I tried to filter IME readings for my flywheel. It confused me, so I never really looked into it much. I also had a gyro on the robot, but the only thing it was used for was for turning 90 degrees twice in programming skills, so no filtering was needed there (we got a jaw-dropping 176 at state :P).

    This year, I figured I should use a gyro. I looked through jmckinney's filtering, and I used it for my robot. The turns were solid with no noticeable drift (compared to a lot of drift without filtering). I only had a couple days so I couldn't make a good programming skills though.

    I've only used quad encoders, potentiometers, and gyros in competition, and the only thing I've filtered are gyro values. I've used IMEs too, but they don't count. I don't know if a lot of teams use filters. It's probably somewhere near the middle of a lot of teams and a few sophisticated programmers.

    Filtering is pretty simple to implement in other algorithms, since it's usually just substituting the filtered value in for the sensor value.

  3. @Vex 9185 I have some limited experience with filters.

    My first experience was back in Nothing But Net, when I tried to filter IME readings for my flywheel. It confused me, so I never really looked into it much. I also had a gyro on the robot, but the only thing it was used for was for turning 90 degrees twice in programming skills, so no filtering was needed there (we got a jaw-dropping 176 at state :P).

    This year, I figured I should use a gyro. I looked through jmckinney's filtering, and I used it for my robot. The turns were solid with no noticeable drift (compared to a lot of drift without filtering). I only had a couple days so I couldn't make a good programming skills though.

    I've only used quad encoders, potentiometers, and gyros in competition, and the only thing I've filtered are gyro values. I've used IMEs too, but they don't count. I don't know if a lot of teams use filters. It's probably somewhere near the middle of a lot of teams and a few sophisticated programmers.

    Filtering is pretty simple to implement in other algorithms, since it's usually just substituting the filtered value in for the sensor value.

    Are you able to provide a link to where we can find jmckinney's filtering. I would like to take a look at it myself as gyro drift is our biggest problem.

  4. Vex 9185

    Mar 21 Bellefontaine, Ohio

    https://www.vexforum.com/index.php/14219-qcc2-gyro-and-pid-control-code/0
    https://bitbucket.org/VRCNERD/nerd_robotc

    PID, motion planning in addition to gyro filtering.

  5. Ashwin Gupta

    Mar 21 California
    Edited 3 months ago by Ashwin Gupta

    For accelerometer specifically, a median filter is very easy to implement and understand compared to Kalman filter and it produces great results usually. The main disadvantage of this is it reduces the resolution at which you can read accelerometer data unless you do a rolling data set and filter on each iteration. Regardless, a median filter is much better than no filter at all because accelerometer results are generally too noisy to work with raw.

    https://en.wikipedia.org/wiki/Median_filter

  6. kypyro

    Mar 21 V5 Beta Tester Central Kentucky
    Edited 3 months ago by kypyro

    @Vex 9185 ...
    This year, I figured I should use a gyro. I looked through jmckinney's filtering, and I used it for my robot. The turns were solid with no noticeable drift (compared to a lot of drift without filtering). I only had a couple days so I couldn't make a good programming skills though....

    @The Electrobotz Are you able to provide a link to where we can find jmckinney's filtering. I would like to take a look at it myself as gyro drift is our biggest problem.

    @jmmckinney reports that compared to the built in capability in RobotC, his code increases drift over time but seems to improve turn measurement accuracy. He said:

    I measured the difference between my code and ROBOTC's filtering for gyros, and I actually found that ROBOTC's won't drift at all if the robot sat still, but would tend to have a less accurate angle measurement after a turn or move, whereas mine would drift something like +/-0.2 degrees over a few minutes but gave great results for turns or other moves.

    Here's his post in context
    https://www.vexforum.com/index.php/32352-gyroscope-drive-straight/p1#p261145

    He's made his source available at the link indicated in @Vex 9185's post above. You can see by reading the code how his filtering works. Computing the average of 16 samples, though that's configurable. Then the average sample is only accepted if it is within 5 sigmas of the no-movement (or calibration) data. If you know some statistics, you can tell what assumptions he's made about the nature of the noise.

  7. callen

    Mar 22 Braintree, MA, USA

    @Ashwin Gupta For accelerometer specifically, a median filter is very easy to implement and understand compared to Kalman filter and it produces great results usually. The main disadvantage of this is it reduces the resolution at which you can read accelerometer data unless you do a rolling data set and filter on each iteration. Regardless, a median filter is much better than no filter at all because accelerometer results are generally too noisy to work with raw.

    https://en.wikipedia.org/wiki/Median_filter

    I agree with median filters in many circumstances. The problem with a lot of filters is that they can get skewed by huge values instead of utterly ignoring them. Filters that are designed to get rid of small amounts of noise may be inappropriate because frequently we're worried about a single stray huge or tiny value showing up and wanting to ignore it. Median filters handle that very simply, as @Ashwin Gupta said. That was what I had a student do when using an ultrasonic rangefinder with an Arduino (seeing values like 30, 31, 30, 2000, 30, 31, 30, 30, ... with a stationary target and robot), and it worked easily and beautifully. We stored the prior two values (@Ashwin Gupta 's mention of a rolling data set) to avoid reducing resolution as much.

    When you're doing things like

    while(rangeFinderValue<target){
    stuff
    }

    you can even forget medians and just check two values, having recorded the prior value. For example,

    while(exitCase==false){
    stuff;
    priorValue=newValue;
    newValue=rangeFinderValue;
    if(priorValue<target && newValue<target){
    exitCase=true
    }
    }

  8. tabor473

    Mar 22 V5 Beta Tester OYES, WPI

    @callen I agree with median filters in many circumstances. The problem with a lot of filters is that they can get skewed by huge values instead of utterly ignoring them. Filters that are designed to get rid of small amounts of noise may be inappropriate because frequently we're worried about a single stray huge or tiny value showing up and wanting to ignore it. Median filters handle that very simply, as @Ashwin Gupta said. That was what I had a student do when using an ultrasonic rangefinder with an Arduino (seeing values like 30, 31, 30, 2000, 30, 31, 30, 30, ... with a stationary target and robot), and it worked easily and beautifully. We stored the prior two values (@Ashwin Gupta 's mention of a rolling data set) to avoid reducing resolution as much.

    When you're doing things like

    while(rangeFinderValue<target){
    stuff
    }

    you can even forget medians and just check two values, having recorded the prior value. For example,

    while(exitCase==false){
    stuff;
    priorValue=newValue;
    newValue=rangeFinderValue;
    if(priorValue<target && newValue<target){
    exitCase=true
    }
    }

    I have done median of last 5 ultrasonic values in the past. It cleans up the signal quite nicely.

  9. nenik

    Mar 22 V5 Beta Tester

    @biglesliep how much improvement are we talking about compared to no filter?

    Could be life-and-death. Take potentiometers - the favorite sensor of my team (they use pots everywhere), but also one that causes most troubles, since it is noisy in a specific way. When turning, it sometimes reports a single value significantly out of line due to the bumpy nature of the spring contact inside. If you use the unfiltered value in a waitUntil()-kind of case, you're bound to fail. Imagine a MoGo down routine that thinks the MoGo is already down because the pot reported one <800 sample among all those 2300 samples. Now they do a primitive "average 3 samples spaced by few milliseconds", which works for a moving target and saves them most of the time. But better approach would be a different kind of filtering - dismissing unlikely values (out of range reading, change too abrupt to by physycally possible, ...)

    @biglesliep do a lot of teams do this, or just a few with really sophisticated programmers?

    Hard to tell unless a lot of teams reply here.

    @biglesliep how do you even know if you need to filter?

    Observing failure modes and ... wait for it ... OK, yes, it is datalogging...

    @biglesliep how hard is it to combine filtering with other algorithms, like PID?

    Not a all. Depending on how you do the filtering (in-line or having a data-collection task, for example), it should pretty much be a matter of calling a function that provides the filtered value instead of accessing the sensor directly.

  10. I have studied @jmmckinney 's code for quite a while now and I am trying to understand how it will prevent gyro drift. For that matter, I don't really even understand what exactly the cause of gyro drift is. My understanding is that due to the vibration caused by motors and other factors, the gyro reads extra turning than is actually happening. How would one go about preventing this? Is someone able to explain this to me like I'm a 10 year old who doesn't understand anything.

    I understand using filters such as a median filter on potentiometers. I believe the concept is taking several different values of the pot and, sorting those values into an ordered list, and then use the media in whatever control loop is being used. However, I don't see how anything similar to this would help prevent gyro drift. I've even read articles on Kalman filters and several other filter types, and have read several other threads about this, but they all seem to go way past my level.

    I would really appreciate a babied down explanation.

  11. jmmckinney

    Apr 14 V5 Beta Tester Cambridge, MA QCC2
    Edited 3 months ago by jmmckinney

    @The Electrobotz I have studied @jmmckinney 's code for quite a while now and I am trying to understand how it will prevent gyro drift. For that matter, I don't really even understand what exactly the cause of gyro drift is. My understanding is that due to the vibration caused by motors and other factors, the gyro reads extra turning than is actually happening. How would one go about preventing this? Is someone able to explain this to me like I'm a 10 year old who doesn't understand anything.

    I understand using filters such as a median filter on potentiometers. I believe the concept is taking several different values of the pot and, sorting those values into an ordered list, and then use the media in whatever control loop is being used. However, I don't see how anything similar to this would help prevent gyro drift. I've even read articles on Kalman filters and several other filter types, and have read several other threads about this, but they all seem to go way past my level.

    I would really appreciate a babied down explanation.

    Gyro drift is just a side effect of how the sensor operates. The sensor is a mems gyroscope, which measures forces from the Coriolis effect in order to determine the rate at which it is rotating. In order to do this, the sensor actually vibrates itself. You can probably assume that some amount of noise is guaranteed to be introduced from that process. In order to get accurate estimates of angular measurements, some filter(s) are necessary.

    You really don't need a complex filter to get good data out of this sensor, just a little bit of understanding of how it works and what you're actually trying to get it to do.

    First thing to know: the gyro sensor itself does not directly measure the angle in which it rotates. It measures the rate at which it rotates over time. When you declare a sensor port as a gyro in ROBOTC, it will apply some logic for you in order to transform its raw measurements to a measured angle of rotation. My code replaces the ROBOTC functionality in an attempt to give more accurate measurements, but does not add up the angular rate for you, I assume that the user will be doing that. This allows users to do more filtering and/or estimation or sensor fusion, whether it be a low pass filter, kalman filter, median filter, etc...

    The way my code does this filtering is by estimating what the raw value of the sensor is supposed to be at zero movement (which ends up being centered around 1850 +/- a bit), and discarding any values which fall into a specified range of that zero movement value (I assume that the distribution of noise is Gaussian, and discard anything within N standard deviations, with N=5 being the default). Conceptually, the robot is interpreting very slow rotation/movement at no motion, therefore drifting less or not at all, while faster motion is treated normally, which is why turns are still accurate. Think of N as a sensitivity parameter, higher values of N mean that the sensor is less sensitive to movement, while lower values would be more sensitive but subject to drift.

    The zero movement value and standard deviation are calculated in a calibration routine that runs when the user calls gyroInit().

  12. @jmmckinney Gyro drift is just a side effect of how the sensor operates. The sensor is a mems gyroscope, which measures forces from the Coriolis effect in order to determine the rate at which it is rotating. In order to do this, the sensor actually vibrates itself. You can probably assume that some amount of noise is guaranteed to be introduced from that process. In order to get accurate estimates of angular measurements, some filter(s) are necessary.

    You really don't need a complex filter to get good data out of this sensor, just a little bit of understanding of how it works and what you're actually trying to get it to do.

    First thing to know: the gyro sensor itself does not directly measure the angle in which it rotates. It measures the rate at which it rotates over time. When you declare a sensor port as a gyro in ROBOTC, it will apply some logic for you in order to transform its raw measurements to a measured angle of rotation. My code replaces the ROBOTC functionality in an attempt to give more accurate measurements, but does not add up the angular rate for you, I assume that the user will be doing that. This allows users to do more filtering and/or estimation or sensor fusion, whether it be a low pass filter, kalman filter, median filter, etc...

    The way my code does this filtering is by estimating what the raw value of the sensor is supposed to be at zero movement (which ends up being centered around 1850 +/- a bit), and discarding any values which fall into a specified range of that zero movement value (I assume that the distribution of noise is Gaussian, and discard anything within N standard deviations, with N=5 being the default). Conceptually, the robot is interpreting very slow rotation/movement at no motion, therefore drifting less or not at all, while faster motion is treated normally, which is why turns are still accurate. Think of N as a sensitivity parameter, higher values of N mean that the sensor is less sensitive to movement, while lower values would be more sensitive but subject to drift.

    The zero movement value and standard deviation are calculated in a calibration routine that runs when the user calls gyroInit().

    Sorry for the delayed response. I've been really busy with exam prep and whatnot. Anyways, this is very helpful and I really appreciate you writing this for me.

 

or Sign Up to reply!