Yes, general computers are fundamentally unsafe. We should always think about threat models, vulnerabilities, blast radii, defense in depth.
What we should never do is dismiss something as a non-concern because we don't know how it could be a problem. Especially when someone is trying to point out something we're doing is extensively documented as a security concern. In that case it would be quite obtuse to claim in a public discussion that the person pointing it out is wrong because you don't understand the issue, and yet I have lived through that.