More pressingly - how can a doctor provide the best care in the absence of records?
Imagine trying to ship a bugfix for a complex system where you don't have the source code and the business owner is refusing to give you the documentation but will sue you and ruin your career if you make a mistake.
I'd probably refuse to see the patient given how much pressure doctors are under to not make a mistake.
What happens if you got a nasty gash and wanted the wound treated and the doctor wanted to issue antibiotics as a precaution but you had a penicillin allergy (which you either forgot to disclose or simply didn't know about but were tested for at some stage in your childhood)
you shouldn't be. to a certain group of people, it is particularly loathsome to be prohibited from doing things that can only hurt the person doing them. this is basically how we treat children.
a patient should be allowed to limit access to their medical records for pretty much any reason. an individual doctor should also be allowed to refuse to treat a patient who is unwilling to disclose part or all of their medical records; they may have valid liability concerns. but revealing your records should not be a precondition to accessing the entire field of medicine. what part of this do you have a problem with? you still get to deal with your doctors however you want.