There is a way to do it, but it’s not the most typical way MacOS users do it for everything else, which involves Right Click->Open With->Other->Always Open With. Xcode’s file associations are super aggressive.
I believe that “Get Info”->”Open With”->”Change All…” still works, and there are command line methods or third party tools.
This has driven me to madness too.