What about the fact that you are moving faster when on top of a mountain than in a valley due to the rotation of the earth and being farther from the center of rotation?
Does this also have an effect on your relative time?
Yes, it makes the equipotential surfaces of Earth's gravitational field (the surfaces on which time ticks "at the same rate") ellipsoids instead of spheres. The "geoid", which is the standard such surface that defines UTC on Earth, is the equipotential surface that averages to the Earth's sea level, and is 13 miles further from Earth's center at the equator vs. the poles.