Using Nginx to load balance microservices (opens in new tab)

(hagbarddenstore.se)

38 pointshagbarddenstore10y ago10 comments

10 comments

"highly available web applications."

but nginx will drop traffic during reloads...

jhgg10y ago

This is false. Nginx will spawn new workers with the changed config and gracefully shutdown the old ones once the new ones are online. There should be no drop in traffic or dropped requests while this happens.

skinowski10y ago

Before making this statement, think of how the following states can really be non-disruptively handled:

1) requests in transit to nginx (in-flight) 2) TCP handshakes just completed (not yet accepted) 3) TCP handshakes in progress (clients will retry if server doesn't RST)

Nginx cannot know if requests are in transit nor it can handle just established TCP conns. It also does not drain the backlog queue in the old processes. It simply tears down estimated 'idle' connections.

Try a test with heavy load and you will see. More network latency more errors...

That's exactly why Nginx Plus offers dynamic upstream modification features as opposed to relying on reload.

paulddraper10y ago

Really? That would mean there's little difference between reload and restart.

skinowski10y ago

Reload runs old/new processes in parallel, so switch is super quick and new connections always land on new processes. So error rate between reload and restart is huge.

hagbarddenstoreOP10y ago

I haven't seen this behaviour, got any proof of your claims?

skinowski10y ago

Any load tester will do the trick. Network latency and client keepalives enabled will also increase likelihood of errors.

Dolores1210y ago

did you consider haproxy?

hagbarddenstoreOP10y ago

In the cases I use this setup, HAProxy won't fit, since I route based on URL values. I do have HAProxy in front of my nginx machines though.

lsaferite10y ago

I actually use this exact set of software.

etcd -> confd -> HAProxy

It works great. I can dynamically route incoming traffic to groups of endpoints based on hostname.

j / k navigate · click thread line to collapse

10 comments

skinowski10y ago

"highly available web applications."

but nginx will drop traffic during reloads...

jhgg10y ago

skinowski10y ago

Before making this statement, think of how the following states can really be non-disruptively handled:

1) requests in transit to nginx (in-flight) 2) TCP handshakes just completed (not yet accepted) 3) TCP handshakes in progress (clients will retry if server doesn't RST)

Try a test with heavy load and you will see. More network latency more errors...

That's exactly why Nginx Plus offers dynamic upstream modification features as opposed to relying on reload.

paulddraper10y ago

Really? That would mean there's little difference between reload and restart.

skinowski10y ago

Reload runs old/new processes in parallel, so switch is super quick and new connections always land on new processes. So error rate between reload and restart is huge.

hagbarddenstoreOP10y ago

I haven't seen this behaviour, got any proof of your claims?

skinowski10y ago

Any load tester will do the trick. Network latency and client keepalives enabled will also increase likelihood of errors.

Dolores1210y ago

did you consider haproxy?

hagbarddenstoreOP10y ago

In the cases I use this setup, HAProxy won't fit, since I route based on URL values. I do have HAProxy in front of my nginx machines though.

lsaferite10y ago

I actually use this exact set of software.

etcd -> confd -> HAProxy

It works great. I can dynamically route incoming traffic to groups of endpoints based on hostname.

j / k navigate · click thread line to collapse