Try disabling rerouting for Geolocation requests to GrapheneOS, because I personally found the gps provider integrated in it to be almost unusable, since it doesn't implement Bluetooth/wifi scanning at all.
Yes that means google will get your location, but it's still better than going back to stock which is also better than any other third party skin in terms of privacy/security.