I wrote a note to the kids in my blog re why the cortex outputs 253 with an open A/D port. I did the calculation assuming a pull up resistor 470k, a pull down resistor of 20K and a series resistor of 10 k per the cortex interface spec.
So with no input, we would expect a digital value of 5v*(20k)/(470k+20k) *4096/3.3v=253 . This is just ohms law applied to series resistors working as a voltage divider. This is indeed close to what I see using the debug with a potentiometer sensor type.
With a plug in , the debug sensor value is around 5 counts.
With the input shorted to ground, the 10K series resistor is now in parallel with the 20K pull down resistor. This gives an equivalent pull down of 6.67k ohms. When I calculate the counts 5v*6.67K/(470k + 6.67k)*4096/3.3v = 87 . So I’m not sure why the Cortex outputs 5. Any ideas? Did I calculate correctly?
My speculation:
It may be that there is some software compensation that attempts to linearize the counts due to the fact that the series and pull-down resistors are about the same magnitude of the Vex potentiometer (10k) and they load the pot down so the actual voltage is nonlinear with pot rotation. One can derive the formula for the exact counts using circuit theory. If you ignore the 470k pull-up resistor (reasonable for most pot resistances) then the counts are given by
counts = 40965/3.3( p2)/((1-p)(p+ 3) + 3*p) where p is the pot ratio [0 to 1]
If p = 1 ( pot outputs 5 volts) counts = 4096*10/9.9 or about full value
If p= 0 (pot outputs 0 volts) counts = 0
if p = .5 counts = 4096* 5/3.31/3.25= 4096( .466)
The .466 would be .5 if the pot wasn’t loaded down. So it is about a 6% error. Correction software could use the inverse of the above counts eq and solve for p given counts. This would be the real ratio for a given count.