It would be a loss if due to a critical need to protect privacy we end up inadvertently harming a group of people that are already at a disadvantage. I once knew a hearing-disabled person and the hardship it brought them affected nearly every interaction with them I had. For me that trade-off is not worth it.
If the best channel to help the deaf listen clearly is also the best one for letting eavesdroppers listen clearly, then this is a problem best not handled with such a heavy handed solution - possibly not even a technological solution at all.
