I thought it was reasonably secure (consider this would only include minimal binaries required for rsync without an interactive shell), but I am relatively ignorant in this area, would you care to elaborate?

LXC/D was on my radar but all of this is a little more complexity than I was anticipating dealing with myself (the security side of locking down the rsync account at least). I may even settle for "restricted rsync". This is why I value rsync.net, they do this for me - and for a much lower cost I might add (I am not a security expert).