undefined | Better HN

0 pointssinghrac5y ago0 comments

I've heard credible claims that GPUs these days (esp. TPUs) have lower latency for big models than CPUs. I haven't really investigated, but I could see it happening if you give the TPU a huge L1 cache or something.

0 comments

someguydave5y ago

Perhaps for large calculations? Otherwise the PCI transfer delay would be a big latency hit?

adamnemecek5y ago

Yeah until TPUs can directly communicate with the sound card, it sounds slow.

j / k navigate · click thread line to collapse

0 pointssinghrac5y ago0 comments

0 comments

someguydave5y ago

Perhaps for large calculations? Otherwise the PCI transfer delay would be a big latency hit?

adamnemecek5y ago

Yeah until TPUs can directly communicate with the sound card, it sounds slow.

j / k navigate · click thread line to collapse