undefined | Better HN

0 pointsMarkSweep9y ago0 comments

I think IO Completion Ports [1] in Windows are pretty similar to kqueue [2] in FreeBSD and Event Ports [3] in Illumos & Solaris. All of them are unified methods methods for getting change notifications on IO events and file system changes. Event Ports and kqueue also handle unix signals and timers.

Windows will also take care of managing a thread pool to handle the event completion callbacks by means of BindIoCompletionCallback [4]. I don't think kqueue or Event Ports has a similar facility.

[1]: https://msdn.microsoft.com/en-us/library/windows/desktop/aa3... [2]: https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2 [3]: https://illumos.org/man/3C/port_create [4]: https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...

0 comments

trentnelson9y ago

BindIoCompletionCallback is very old, the new threadpool APIs should be used, e.g.: https://github.com/pyparallel/pyparallel/blob/branches/3.3-p...

Regarding the differences between IOCP and epoll/kqueue, it all comes down to completion-oriented versus readiness-oriented.

https://speakerdeck.com/trent/pyparallel-how-we-removed-the-...

MarkSweepOP9y ago

The documentation for the new thread pool APIs in Windows say that it is more performant than the old ones. Do you have any experience transitioning from the old thread pool to the new one, and if so did you notice performance differences?

dblohm79y ago

Last decade I wrote a server based on the old API and BindIoCompletionCallback. I can't comment on the performance differences, but the newer threadpool API is much better to work with.

IMO one of the biggest problems with the old API is that there is only one pool. This can cause crazy deadlocks due to dependencies between work items executing in the pool. The new API allows you to create isolated pools.

The old API did not give you the ability to "join" on its work items before shutting down. You could roll your own solution if you knew what you were doing, but more naive developers would get burned.

You also get much finer grained control over resources.

trentnelson9y ago

I'm really lucky to have decided to dig into this stuff (for PyParallel) around ~2011, by which stage Windows Vista (where all the new APIs had been introduced) had been out for like, 5-6 years, so no, I never had to deal with the old APIs, thankfully.

I really sympathize with Len Holgate though... he's had to nurse Server Framework through all of that transition, which must have been painful: http://www.serverframework.com/asynchronousevents/2016/06/an...

j / k navigate · click thread line to collapse

0 pointsMarkSweep9y ago0 comments

Windows will also take care of managing a thread pool to handle the event completion callbacks by means of BindIoCompletionCallback [4]. I don't think kqueue or Event Ports has a similar facility.

0 comments

trentnelson9y ago

BindIoCompletionCallback is very old, the new threadpool APIs should be used, e.g.: https://github.com/pyparallel/pyparallel/blob/branches/3.3-p...

Regarding the differences between IOCP and epoll/kqueue, it all comes down to completion-oriented versus readiness-oriented.

https://speakerdeck.com/trent/pyparallel-how-we-removed-the-...

MarkSweepOP9y ago

dblohm79y ago

Last decade I wrote a server based on the old API and BindIoCompletionCallback. I can't comment on the performance differences, but the newer threadpool API is much better to work with.

You also get much finer grained control over resources.

trentnelson9y ago

j / k navigate · click thread line to collapse