undefined | Better HN

0 pointsnickysielicki11mo ago0 comments

I don’t mean to call you or your pseudocode out specifically, but I see this sort of thing all the time, and I just want to put it out there:

PSA: if you ever see code trying to measure timing and it’s not using the CUDA event APIs, it’s fundamentally wrong and is lying to you. The simplest way to be sure you’re not measuring noise is to just ban the usage of any other timing source. Definitely don’t add unnecessary syncs just so that you can add a timing tap.

https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART_...

0 comments

bee_rider11mo ago

If I have a mostly CPU code and I want to time the scenario: “I have just a couple subroutines that I am willing to offload to the GPU,” what’s wrong with sprinkling my code with normal old python timing calls?

If I don’t care what part of the CUDA ecosystem is taking time (from my point of view it is a black-box that does GEMMs) so why not measure “time until my normal code is running again?”

nickysielickiOP11mo ago

If you care enough to time it, you should care enough to time it correctly.

bee_rider11mo ago

I described the correct way to time it when using the card as a black-box accelerator.

1 more reply

gavinray11mo ago

One of the wisest things I've read all week.

I authored one of the primary tools for GraphQL server benchmarks.

I learned about the Coordinated Omission problem and formats like HDR Histograms during the implementation.

My takeaway from that project is that not only is benchmarking anything correctly difficult, but they all ought to come with disclaimers of:

"These are the results obtained on X machine, running at Y time, with Z resources."

jms5511mo ago

Never used CUDA, but I'm guessing these map to the same underlying stuff as timestamp queries in graphics APIs, yes?

saagarjha11mo ago

I mean you can definitely use it in a pinch if you know what you’re doing. But yes the event APIs are better.

j / k navigate · click thread line to collapse

0 pointsnickysielicki11mo ago0 comments

I don’t mean to call you or your pseudocode out specifically, but I see this sort of thing all the time, and I just want to put it out there:

https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART_...

0 comments

bee_rider11mo ago

If I don’t care what part of the CUDA ecosystem is taking time (from my point of view it is a black-box that does GEMMs) so why not measure “time until my normal code is running again?”

nickysielickiOP11mo ago

If you care enough to time it, you should care enough to time it correctly.

bee_rider11mo ago

I described the correct way to time it when using the card as a black-box accelerator.

1 more reply

gavinray11mo ago

One of the wisest things I've read all week.

I authored one of the primary tools for GraphQL server benchmarks.

I learned about the Coordinated Omission problem and formats like HDR Histograms during the implementation.

My takeaway from that project is that not only is benchmarking anything correctly difficult, but they all ought to come with disclaimers of:

"These are the results obtained on X machine, running at Y time, with Z resources."

jms5511mo ago

Never used CUDA, but I'm guessing these map to the same underlying stuff as timestamp queries in graphics APIs, yes?

saagarjha11mo ago

I mean you can definitely use it in a pinch if you know what you’re doing. But yes the event APIs are better.

j / k navigate · click thread line to collapse