Not that I'd recommend it, but if you symlink your system library into the bazel build area, as long as your sandboxing setup don't hose you (or you just turn it off), bazel will track system tools/library in the same way as everything else.
Bazel's rules_cc even has a system_library.bzl you can import a `system_library` from that automates this for you. https://github.com/bazelbuild/rules_cc/blob/main/cc/system_l...
I'd still recommend building everything from scratch (and understanding the relationships and graph of your dependencies), but if your build isn't that complicated and you want to role the dice on UB, this isn't that hard.
As an aside, the most galling part of bazel's cache key calculations has to be that it's up to the individual rules to implement this how they see fit. The rules native to bazel written in java vary wildly compared to starlark-written rules. On thing you (or someone in your org) end up becoming pretty comfortable with while using bazel in anger is RTFC.