Enough flying phones would cause a significant performance degradation. The issue of handling fast switching was solved, but any "because security" or "because safety" laws tend to stay for a long time.

> There is another reason sometimes cited too for the early introduction of mobile phone restrictions, and it is related to the potential impact on cellular ground equipment.

> A mobile device operating at an altitude, and moving at speed, can see multiple cell towers at the same time. This will block frequencies used by these towers, with much more activity than they were designed to handle from ground-based devices. In the US, the ban on electronic devices in flight was initially put in place by the US Federal Communications Commission (FCC), not the FAA, for this reason.

https://www.faa.gov/travelers/fly_safe/information

> The FCC and FAA ban cell phones for airborne use because its signals could interfere with critical aircraft instruments. Devices must be used in airplane mode or with the cellular connection disabled. You may use the WiFi connection on your device if the plane has an installed WiFi system and the airline allows its use.

You'll note the FCC in there. While the aircraft instrumentation is the most immediate "reason for people to do this" that people can identify, disrupting ground communications is also there. Unfortunately, people are not that likely to do it based on "you'll inconvenience people using the cell phone towers that you fly over."