Signal does indeed use an architecture (at least for chats with contacts, or optionally everyone when you enable the "sealed sender" option that makes you a bit more prone to receiving spam) where Signal doesn't know who's sending a given message from a given IP address, and only which account it's destined for.

But any entity in position to globally correlate traffic flows into and out of Signal's servers can just make correlations like "whenever Alice, as identified by her phone's IP, sends traffic to Signal, Bob seems to be getting a push notification from Apple or Google, and then his phone connects to Signal, so I think they're talking".