In that sense, if a vaccine purely reduced transmission even without offering any other protection, mandating it could still be justified for certain activities, just like a driver's license is considered reasonable today.
I'm sure driver's licenses were considered very controversial restrictions on personal freedom back when they were first introduced.