Was super confused about the leak vector cause I was thinking in HTML terms not javascript. Found a nice paper that illustrates the timing attack
https://link.springer.com/content/pdf/10.1007/978-3-319-1846...
Having read that I'm surprised they haven't done a hybrid approach. store the site key with the cache but also the initial download time. If a site requests a resource that's cached but not for that site don't go download it but artificially delay it's delivery from the cache such that it's indistinguishable from a normal fetch (utilizing the original time maybe with some variance) then log the new site key such that further requests from that site appear to be using an isolated cache.
while you don't get any real first-time performance boost you do save actual bits over the wire so there is real life impact and savings while eliminating the source of the timing attack.
