Do you know what other hardware your baseband processor has the ability to inspect?
Qualcomm SMMU: https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets...
Apple: https://support.apple.com/lt-lt/guide/security/seca4960c2b5/...
Samsung (vuln indicating it wasn’t configured correctly, but they still do have and use an IOMMU): https://nvd.nist.gov/vuln/detail/CVE-2022-39854
The main CPU/application processor/main CPU might be running better secured Unix/Linux and might be able to protect itself from peripheral CPUs, but that's not the point; a phone had always been a pair (minimum) of computers, traditionally referred to as Application Processor(AP) and Baseband Processor(BP), of only the slightly faster one is exposed to the user, and it's unclear what is going on inside the other one or how to handle it. That's the problem.
Encryption only helps if the endpoints that can get access to the plaintext are not compromised.