undefined | Better HN

0 pointsnurettin5y ago0 comments

I have a prototype system where I am passing over 100k msg/sec between a dozen backend services written in python (asyncio+redis) and I keep on wondering when my bottleneck will become functional programming and safe message passing by making copies. When will the madness end?

0 comments

bsaul5y ago

i don't think erlang is requiring FP + immutable + message passing for performance, but rather for correctness. Making copies is actually more expensive in terms of perf.

nurettinOP5y ago

me too, and I have no clue as to how changing language can increase correctness in my case. I can think of other use cases where in-process concurrency and mutating data structures could cause problems, but I avoid those scenarios entirely anyway.

pdimitar5y ago

Tradeoffs. In the setup you describe I'd pick correctness over performance any day.

If you hit a performance bottleneck you might as well just use RabbitMQ or Kafka to queue up stuff and process it as it comes along. Or apply back-pressure if your current code allows for it.

nurettinOP5y ago

I actually use redis hashsets to queue stuff when I need correctness. Much more performant than rmq's mnesia

j / k navigate · click thread line to collapse

0 pointsnurettin5y ago0 comments

0 comments

bsaul5y ago

i don't think erlang is requiring FP + immutable + message passing for performance, but rather for correctness. Making copies is actually more expensive in terms of perf.

nurettinOP5y ago

pdimitar5y ago

Tradeoffs. In the setup you describe I'd pick correctness over performance any day.

If you hit a performance bottleneck you might as well just use RabbitMQ or Kafka to queue up stuff and process it as it comes along. Or apply back-pressure if your current code allows for it.

nurettinOP5y ago

I actually use redis hashsets to queue stuff when I need correctness. Much more performant than rmq's mnesia

j / k navigate · click thread line to collapse