https://www.lasmux.com/product/single-ended-active-probes/

For a real receiver circuit you'd want to sample at least once a symbol obviously (so RX sample rate equals the baud rate). In practice you'd want more than that so you can better do DSP afterwards to remove channel losses and such. For a test instrument, it's going to depend on what exactly you want to see. Most standards define a required filter shape and BW for testing and you need enough sample rate in your scope to be able to implement this (usually in DSP).

Ex:9600 baud FSK signal centered at 440MHz still needs something capable of capturing around 440MHz, but this has nothing to do with baud rate. A 500MHz scope would do well for acquiring the signal.

Practically radios often divide the problem into two parts, first use a mixer to shift the signal down and filter out the parts you don't want, then decode the FSK which remains at a much lower frequency.

This is the governing limit: https://en.m.wikipedia.org/wiki/Shannon%E2%80%93Hartley_theo... and it considers things like SNR as well.