- Shell compiled without built-ins
- No coreutils
- No sudo
- Root account disabled
- Read-only root filesystem
- No user home directories
- Destroyed and rebuilt from template every X hours on some maintenance schedule
Effectively, all you can do is ssh in, ssh out, and forward ports. It might be theoretically possible, but as far as I know, no one has ever compromised one, especially since you can already only get to the bastion from a government VPN anyway, and authentication to that requires a smart card, so there are an awful lot of things you need to compromise to get to that point.
This also answers the suggestion down the page of "why don't you just apply the same controls to every host and not have a bastion." Because the bastion is unusable and you want to actually use your other hosts.