In general you want to have some rubber band tensioning to slightly push tracking wheels into the floor and mount them far from each other to reduce tracking error and jitter.
This topic has a lot of pictures and advice on making good tracking wheels for odometry: Position Tracking Wheels Mechanism
For example, the back view of the 5225A robot:
240P’s robot design was a very well thought through application of tracking wheels:
And this was an example of packaging tracking system in a very tight chassis: