undefined | Better HN

0 pointspgaddict9y ago0 comments

Not really. Recent CPUs (at least those from Intel, which is what EC2 runs on) implement constant_tsc, so the frequency does not affect the tsc.

A worse issue is that the counters may not be synchronized between cpus, which may be an issue when the process moves between sockets.

But I wouldn't call that "dangerous", it's simply a feature of the clock source. If that's an issue for your program, you should use CLOCK_MONOTONIC anyway and not rely on gettimeofday() doing the right thing.

0 comments

blibble9y ago

how does constant_tsc interact with VMs being silently migrated from one physical machine to another?

kondro9y ago

EC2 doesn't do any kind of live machine migration. The only times a machine may start on a different host is if it is stopped and then started. Even reboots don't allow them to move.

You see this a lot when AWS lets you know about maintenance on a physical host and gives you the option to avoid the automated move by doing these steps manually at a time of your choosing before the maintenance window.

pgaddictOP9y ago

Not sure, but it can't be better than moving processes between CPUs I guess. Also, does EC2 silently move VMs like this?

poofyleek9y ago

Even without migration, the synchronization can be an issue. In older multi-core machines, tsc synchronization was an issue among cores. Modern systems take care of this. And core CPU clock frequency change is also taken care of, so that constant rate is available via tsc. However, when hypervisors such as VMWare or paravirtualization like Xen come into play, there are further issues, because RDTSC instruction either has to be passed through to physical hardware or emulated via a trap. When emulated a number of considerations come into play. Xen actually has PVRDTSC features that are normally not used but can be effective in paravirtual environments. The gettimeofday() syscalls (and clock_gettime) are liberally used in too many lines of existing software. Their use is very prevalent due to historical reasons as well as many others. One reason is that the calls are deceptively "atomic" or "isolated" or "self-contained" in their appearance and usage. So liberal use is common. A lot of issues come about due to their use, especially in time sensitive applications (e.g. WAN optimization). This is especially true in virtual environments. There are complex issues described elsewhere that are kind of fun to read. https://www.vmware.com/pdf/vmware_timekeeping.pdf and https://xenbits.xen.org/docs/4.3-testing/misc/tscmode.txt. The issue becomes even more complex in distributed systems. Beyond NTP. Some systems like erlang has some provisions, like http://erlang.org/doc/apps/erts/time_correction.html#OS_Syst.... Other systems use virtual vector clocks. And some systems, like google TrueTime as used in Spanner, synchronize using GPS atomic clocks. The satellite GPS pulses are commonly used in trading floors and HFT software. This is a very interesting area of study.

1 more reply

jdamato9y ago

We don't really know what EC2 does or precisely the type of hardware your VM will be spun up on. I've erred on the side of being cautious due to the vast amount of work being invested in timekeeping in various hypervisors. If EC2 knows that the TSC clocksource is safe on all of its hardware, perhaps modifying the Amazon Linux AMI to set TSC as the default clocksource would reassure many folks, myself included.

Advanced users that can run their own analysis or who have applications which would withstand potential time warps, are of course, free to ignore my warning at their own risk ;)

j / k navigate · click thread line to collapse