I think you're misunderstanding.
"go build" of arbitrary attacker controlled go code will not lead to arbitrary code execution.
If you do "git clone attacker-repo && cargo build", that executes "build.rs" which can exec any command.
If you do "git clone attacker-repo && go build", that will not execute any attacker controlled commands, and if it does it'll get a CVE.
You can see this by the following CVEs:
https://pkg.go.dev/vuln/GO-2023-2095
https://pkg.go.dev/vuln/GO-2023-1842
In cargo, "cargo build" running arbitrary code is working as intended. In go, both "go get" and "go build" running arbitrary code is considered a CVE.