The part you're correct about is that as they control the device, there's nothing saying they can't build a backdoor into it that reports the information (ie. location) back to them once it hits your phone. And we're also taking it on trust that it works the way they say it works, as it's not open-source.

But as someone else commented, eventually you have to trust something.

It's possible in theory they could add hooks into the OS to then do that next time you use the key (e.g. effectively a baked in backdoor) or maybe in some cases the key can be extracted from an iCloud backup or similar (I'm not 100% sure how those keys are stored, but it's likely detailed in their security documentation) but in general the service itself cannot see the encrypted device location.