But being part of an organization, don’t they have admin control over your account? Could delete all of your repos, reset your keys, access private repos, etc.
Even if a tiny risk, it seems silly just to bolster the GH activity graph.
No. They have control over your membership in their org and which of their repos you can access, not your repos. Note that a GitHub account can be members of multiple orgs.