I wonder if a classic-style cameraless HUD could alleviate some of the social issues while still retaining enough of the functionality. I mean it'd still take a certain kind of person to wear it, but people who wouldn't wear it who are actively angry at / frightened of Google Glass, rather than merely indifferent to it, seem to be so mostly because of the possibility that the person wearing it could be recording them at any time, without outwardly visible signs (while recording someone by holding up your smartphone is typically pretty obvious). The HUD itself doesn't seem to particularly anger people.

On the other hand, if some devices come with cameras and others don't, people with the no-recording versions might still suffer the same stigma, if the average person can't tell at a glance which is which.