Data brokers for pharmacy have your prescription data, your doctor is not protected information and other aspects of your care are available to many entities for purposes like insurance subrogation that also create data products.
HIPAA protects you from gossiping clerks at health facilities and HR. It prevents the use of some bad IT practices, and forces you to sign lots of disclosures. That’s about it.
If you want that type of information to be private, don’t collect it or don’t share it with 3rd parties in an accessible form.
