If repos are public, then you must assume that once something is pushed then someone has copied it.
You may get lucky and remove/hide it fast enough, or think you did...
This is an with Github today, all public repos are being watched by bots reviewing all commits for accidentally-pushed credentials.
The only solution is to not use a public repo.