Current AI (the one that's getting regulated, e.g. LLMs and diffusion models) lacks any sort of individual drive or initiative, so all the danger it represents is that of a powerful tool wielded by someone.
People who are saying AI is dangerous are saying people are dangerous when empowered with AI, and that's why only the right people should have access to it (who presumably are the ones lobbying for legislation right now).