ARM has a reference design which includes drivers.
The OEM is responsible for the final design and validation.
ARM cannot take responsibility for validating 1000s of implementations that vary in both hardware and software configurations and validate them for billions of devices.
Neither can the FOSS community.
For the handful of boards which are popular enough to have sufficient tracking for the FOSS community to take the wheel if an open source driver was available the OEM has sufficient incentive to support their platform.
For the other 4,999,999,990 LowHo NiHao industries unbranded boards it wouldn't matter in the first place.