undefined | Better HN

0 pointsachamayou9y ago0 comments

That's why microbenchmarks that have been on the internet for a while are quite nice: there's a good chance at least some experts from each language will have had a go and submitted a decent implementation.

0 comments

rbehrends9y ago

But that's a highly (and narrowly) optimized version that took a lot of effort to build. It's not something that the average programmer you have working for you will be able to replicate. Also, some languages will not have equivalent effort put in the corresponding implementations (simple example: some languages have parallelized solutions, some don't).

nostrademons9y ago

I think the idea behind real-world software performance is that "if you need it, you really need to have it, but most of the time you don't need it." The benchmarks game isn't all that unrealistic for this. Most of the time, you won't care about performance at all. When you do care about performance, you'll be able to devote an experienced engineer to carefully optimize a specific hot spot, not unlike a microbenchmark. It matters how fast you can get the code to go when it's a hot spot, not how fast all your little support code runs.

My one complaint is that there's no benchmark that measures FFI performance. Realistically, if you build a system in Python or Ruby - you're going to be dropping down to C for your hot spots. And so scripting language performance on all these compute-intensive tasks is somewhat irrelevant, you really want to know how much overhead you'll incur crossing the scripting/C boundary (which, in my experience can sometimes be large enough that it wipes out all the gains of coding in C in the first place).

igouy9y ago

> measures FFI performance

Many of the pi-digits programs use GMP.

https://benchmarksgame.alioth.debian.org/u64q/performance.ph...

rbehrends9y ago

I have some experience with codegolfing for speed and in my experience, these benchmarks do not reflect the reality of such problems, either. If you really want to tweak code for speed, you'll use a mix of algorithmic improvements (that's where the biggest gains are) tailored to your language and compiler/hardware, possibly using FFI/assembly if you absolutely need it. But the benchmark game explicitly disallows algorithmic improvements, for example. Except through the backdoor, where language implementors can sometimes tweak libraries to circumvent restrictions.

And to make things more complicated, the performance increase is generally a function of the time spent on optimizing the problem, which is dependent not only on any innate speed, but also the expressiveness of the language and the programmer's familiarity with the language. Given that you don't have infinite time, there are further tradeoffs here.

1 more reply

j / k navigate · click thread line to collapse

0 comments

rbehrends9y ago

nostrademons9y ago

igouy9y ago

> measures FFI performance

Many of the pi-digits programs use GMP.

https://benchmarksgame.alioth.debian.org/u64q/performance.ph...

rbehrends9y ago

1 more reply

j / k navigate · click thread line to collapse