undefined | Better HN

0 pointsfavorited11mo ago0 comments

It's possible that they were referring to something specific about their platform and its system allocator, but like I said it was an anecdote about one engineer's statement. I just remember thinking it sounded fair at the time.

0 comments

vlovich12311mo ago

The “system” allocator is managing memory within a process boundary. The kernel is responsible for managing it across processes. Claiming that a user space allocator is greedily inefficient is voodoo reasoning that suggests the person making the claim has a poor grasp of architecture.

jeffbee11mo ago

There are shared resources involved though, for example one process can cause a lot of traffic in khugepaged. However I would point out that is an endemic risk of Linux's overall architecture. Any process can cause chaos by dirtying pages, or otherwise triggering reclaim.

vlovich12311mo ago

That’s generally true of any allocator and assuming glibc’s behavior would help mitigate this is critically not something kernel engineers design around nor something glibc allocator is trying to achieve as a design goal.

favoritedOP11mo ago

For context, the "allocator engineer" I was talking to was a kernel engineer - they have an extremely solid grasp of their platform's architecture.

The whole advantage of being the platform's system allocator is that you can have a tighter relationship between the library function and the kernel implementation.

vlovich12311mo ago

I’m not generally aware of any system allocator that’s written hand in glove with the kernel’s allocator or somehow interops better for overall system efficiency at the cost of behavior in-app. Care to provide an example?

jdsully11mo ago

The "greedy" part is likely not releasing pages back to the OS in a timely manner.

nicoburns11mo ago

That seems odd though, seeing as this is one of the main criticisms of glibc's allocator.

jeffbee11mo ago

In the containerized environments where these allocators were mainly developed, it is all but totally pointless to return memory to the kernel. You might as well keep everything your container is entitled to use, because it's not like the other containers can use it. Someone or some automatic system has written down how much memory the container is going to use.

3 more replies

j / k navigate · click thread line to collapse

0 comments

vlovich12311mo ago

jeffbee11mo ago

vlovich12311mo ago

favoritedOP11mo ago

For context, the "allocator engineer" I was talking to was a kernel engineer - they have an extremely solid grasp of their platform's architecture.

The whole advantage of being the platform's system allocator is that you can have a tighter relationship between the library function and the kernel implementation.

vlovich12311mo ago

jdsully11mo ago

The "greedy" part is likely not releasing pages back to the OS in a timely manner.

nicoburns11mo ago

That seems odd though, seeing as this is one of the main criticisms of glibc's allocator.

jeffbee11mo ago

3 more replies

j / k navigate · click thread line to collapse