Couldn't you just pick a pulsar or something, all agree to take their measurements from X distance from pulsar, and use that as the standard? It would be time + location, making your local time depending on the distance and number of pulses from the 0 starting pointing. As long as you kept account of the number of pulses as you moved through space, you could use that as a standard, and whenever you met up somewhere else in the universe, you could base your time based on their observed number of pulses.

Im not sure pulsars are actually that precise and predictable to not require regular adjustments to equate local times, but it would provide a standard anybody could utilize as long as they could keep an eye on it.

Of course it would break down if someone broke light speed, either with wormholes or some unknown technology, but as long as you are below lightspeed then there shouldn't be any problems.