In the usual secondary sequence, Geometry is between Algebra I and Algebra II; doing it in 9th grade, even with a full year precalc after Algebra II, gets you to Calculus I in 12th. with a combined Algebra II/Trig and no separate precalc, which has long been thr common accelerated course, it gets you to Calculus II.
Physics in high school is typically non-calculus based and works well alongside precalc or calc I or even Algebra II; there’s no difficulty having it in 11th/12th with geometry in 9th.
It may create problems keeping mathematically advanced students engaged, it doesn’t create problems teaching algebra or physics.