As part of the wheel placement, you would tend to place the trackers on small arms that hang down so that they maintain constant contact with the field. Also, since they spin freely other than the shaft encoder, there should be no slippage. Therefore, there should be no mechanical slop.
Further, if you do your trig well and use precise functions, you should be able to get your exact position within millimeters. With that in mind, you can work the equations backwards to teach the robot how to move to a specified place, which would be within millimeters. Personally, my user error in measuring is greater than the distance from actual position to intended position.
Then, if you want to take it another step, you could have the function update along the way, as in, if it got pushed out of line or something else goes badly, the robot will correct itself and continue to the desired destination.
TLDR: Yes, if done well, you could have a virtually perfect auton. However, I would suggest due to driver control errors, just code it instead of tracking the driving. It will be far more precise.
I hope that made sense.