From your example: I am from a vulnerable group and I am uncomfortable handing over my phone to authorities. If I am uncomfortable with handing over my phone, why would I digitize my license where I would then be forced to do so? If I was looking for an alternative solution to this problem, it would need to be more convenient than just carrying my existing ID, since the whole advantage of digitizing my ID is so that I don't have to risk losing it. If I have to carry a separate device, that advantage is lost.

Also I would actually talk to the people in the vulnerable groups to validate these assumptions before getting too invested in the narratives. They may not be as concerned as you think or may not have considered the risks.