Apple already has a kind of "backdoor": they store the keys for encrypted cloud backups in their cloud as well. They advertise that cloud data are encrypted but prefer not to mention that they also have a key to decrypt it. Even with the highest level of security [1] your contacts list in Apple Cloud are not encrypted. Why? Probably someone asked for this.
CSV or PNG weren't created with encryption in mind, but one can easily encrypt them. Apple can always make their own proprietary protocol. This doesn't explain anything. However the version that the govt wants to be able to see who is in person't contact list explains it well.