It protects against extraction, not usage on the machine itself. Of course they can use the secret on the compromised machine.
> And although that's a bit limiting, once they've SSHed to a host as me once they can add their own key to authorized_keys in many cases.
Assuming they can edit the file.
> That's why Yubikeys and U2F keys and suchlike have a physical button.
The TPM spec has a policy setup to account for some fingerprint reader that can be used to authenticate. I haven't been able to figure out how/what/whys of the implementation here but this is very much a thing.