And a remote adversary (who sees your VPS IP) cannot know who is, in turn, connected to that VPS, unless they can extract that information from the VPS provider via court order or hacking. [This is of more concern if you plan to do illegal things or are a dissident].

As always, you must model your threat scenario and proceed accordingly.