You can pass it back into Go for dereferencing, at which point you can use the uintptr => unsafe.Pointer map, and it will be correct since the moving GC will have preserved the unsafe.Pointer address at the right location.
The pattern is useful. It just doesn't handle the most common case, which is passing a buffer or a simple output parameter into a C function.