Unsafe blocks are common when interfacing with C or C++ code.
The first thing you want to ensure is that you avoid unsafe in YOUR code. That is, after all, where you are going to create your bugs.
Ideally, there is no unsafe code in the crates you use. If there is, at least there are more eyes on that code to catch problems. You can inspect and debug that code, and others can do it for you. Others may battle test it and stress it far more than you will.
And at least the unsafe code is isolated to a few areas in the supporting libraries. Less to inspect. Less to debug. Fewer places to introduce memory related regressions.
It does not at all follow that, given an unsafe block, we may just have well used C++.