The kernel side of drivers is already published under a FLOSS license, it's just that the code quality is usually subpar and the important changes are crammed into a tarball together with (sometimes) millions of other lines of changed code.

The sources for the matching userspace binaries (which are usually the issue for Android version bumps) are usually under NDA by the component manufacturer and can not be released by the OEM independently.