The specialized sub-processors are implemented differently, not always available, and not available 1:1 to compute nodes. If you're offering, for example, cloud compute... you can offer 4 cores pretty easily... but if there are 2 specialized sub-processors, then do you offer them, does this queue across all users/clients on that system or do you just block and pretend it doesn't exist. For Zen 4c, they're all general compute.
This means, practically speaking, you're only going to really have 1 host on a server that wants/needs these specialized sub-processors. Which means more space/heat/power for a single user/service. It's probably fine for some things, but far from ideal. This also doesn't get into software optimization and alternative paths where not available.
This gets far worse in the ARM space, as it seems every SOC does something different, which means it's often broken, or unusable if you're using a mainline OS/kernel and even then most software won't be optimized for it. At best, you can maybe playback 4k compressed video. At worst, you can't at all. Just speaking to the most common instance in that space, which is video compression, which is often built around closed drivers that mainline OSes (Ubuntu Server, Debian, etc) don't have in the box, and the vendor only supports a single version of a distro fork with no upgrade path.