For this type of attack to work, the algorithm being run needs to be very well understood, and the runtime of the algorithm needs to depend almost entirely on the secret key.
In contrast, the timing of virtually any email operation is not dependent on the contents of the email, other than the size. That is, whether you wrote "my password is hunter2" or "my password is passwor", the timing of any operation running on this email will be identical.