Using Redis SORT and GET to save on roundtrips (opens in new tab)

(trentstrong.com)

64 pointstrentonstrong14y ago8 comments

8 comments

no need to sort by a non existing key, all you need is that the BY pattern does not contain an asterisk, so it is constant, that means all the elements have the same value for sorting, and Redis knows that it can skip it and use the List natural ordering.

It is interesting that you can ask for single elements of an hash, like in: GET object:*->field

With 2.6 and scripting this kind of optimizations will become the rule.

trentonstrongOP14y ago

Ah no asterisk means nothing to replace, makes sense.

Excited for 2.6 and scripting. Has much changed on the scripting side since the 2.2-scripting branch?

ricardobeat14y ago

Good point, but surely using a key like nosort makes the intent clearer.

salimane14y ago

this assume you're connecting to one redis server and not running a cluster of redis servers, or have tagged the keys being requested in a cluster of redis servers

antirez14y ago

When you are connected to multiple servers the best thing is to have a client that is able to parallelize requests against multiple servers.

There is a trick, (used by some memcached client as far as I know), to do that without multiplexing, that is, usually you should do something like this:

  send GET foo to socket1
  send GET bar to socket2
  replies = some_form_of_multiplexing(socket1,socket2)

Instead since we can observe that usually the round trip time between the peers at socket1 and socket2 are similar, we can do:

   send GET foo socket1
   send GET bar socket2
   reply1 = read(socket1)
   reply2 = read(socket2)

This makes the implementation simpler without making the performances so much worse.

So a good Redis client for a sharded environment should be able to do something like this:

  replies = parallel_queries(["GET","foo"],["GET","bar"],["INCR zap"])

Trying to send the queries to all the servers in parallel and returning the replies ASAP, and in the right order of course.

salimane14y ago

@antirez, i see what u mean but in this context with "SORT BY GET", in the case of a single server, the command is send to it in one call and the server replies with the result. but in the case of a cluster, the benefit of having the server handle the one call is no more there as of 2.4 at least. the most a good client could actually do is to take the one call "SORT BY GET", recognize it, divide it, do separate calls, merge the results and send them back...which is going back to what were trying to avoid... or is there another way to handle this currently with 2.4?

est14y ago

http://redis.io/commands/sort

Learned so much today!

durbin14y ago

good point.

j / k navigate · click thread line to collapse

8 comments

antirez14y ago

It is interesting that you can ask for single elements of an hash, like in: GET object:*->field

With 2.6 and scripting this kind of optimizations will become the rule.

trentonstrongOP14y ago

Ah no asterisk means nothing to replace, makes sense.

Excited for 2.6 and scripting. Has much changed on the scripting side since the 2.2-scripting branch?

ricardobeat14y ago

Good point, but surely using a key like nosort makes the intent clearer.

salimane14y ago

this assume you're connecting to one redis server and not running a cluster of redis servers, or have tagged the keys being requested in a cluster of redis servers

antirez14y ago

When you are connected to multiple servers the best thing is to have a client that is able to parallelize requests against multiple servers.

There is a trick, (used by some memcached client as far as I know), to do that without multiplexing, that is, usually you should do something like this:

  send GET foo to socket1
  send GET bar to socket2
  replies = some_form_of_multiplexing(socket1,socket2)

Instead since we can observe that usually the round trip time between the peers at socket1 and socket2 are similar, we can do:

   send GET foo socket1
   send GET bar socket2
   reply1 = read(socket1)
   reply2 = read(socket2)

This makes the implementation simpler without making the performances so much worse.

So a good Redis client for a sharded environment should be able to do something like this:

  replies = parallel_queries(["GET","foo"],["GET","bar"],["INCR zap"])

Trying to send the queries to all the servers in parallel and returning the replies ASAP, and in the right order of course.

salimane14y ago

est14y ago

http://redis.io/commands/sort

Learned so much today!

durbin14y ago

good point.

j / k navigate · click thread line to collapse