Whole numbers can be defined and proven to be necessary to describe the world pretty easily. From there, rational numbers are trivial to define. Negative numbers are somewhat more abstract, but they have very intuitive definitions in many domains, such as accounting. It may be possible to avoid them in a theory of physics, though.

The complex numbers (well, at least those with a rational imaginary part and a rational real part) have been recently proven to be necessary to describe the universe[0] (assuming quantum theory is correct).

The irrational numbers are then are the only numbers that are harder to pin down, and I'm not sure that there is a way to prove that any physical quantity has an irrational value, vs a rational value that is arbitrarily close to that irrational value.

[0] https://arxiv.org/abs/2101.10873