4800kHz and 44100kHz devices appeared at roughly the same time. Sony's first 44100kHz device was shipped in 1979. Phillips wanted to use 44.0kHz.
If you can do 44.1khz on an NSTC recording device, you can do 44.0khz too. Neither NTSC digital format uses the fully available space in the horizontal blanking intervals on an NTSC VHS device, so using less really isn't a problem.
Why is 44Khz better? There's a very easy way to do excellent sample rate conversions from 44.0Khz to 48Khz, you upsample the audio by 12 (by inserting 11 zeros between each sample), apply a 22Khz low-pass filter, and then decimate by 11 (by keeping only every 11th sample. To go in the other direction, upsample by 11, filter, and decimate by 12. Plausibly implementable on 1979 tech. And trivially implementable on modern tech.
To perform the same conversion from 44.1kHz to 48kHz, you would have to upsample by 160, filter at at a sample rate of 160x44.1kHz, and then decimate by 147. Or upsample by 147, filter, and decimate by 160. Impossible with ancient tech, and challenging even on modern tech. (I would imagine modern solutions would use polyphase filters instead, with tables sizes that would be impractical on 1979 VLSI). Polyphase filter tables for 44.0kHz/48.0kHz conversion are massively smaller too.
As for the prime factors... factors of 7 (twice) of 44100 really aren't useful for anything. More useful would be factors of two (five times), which would in increase the greatest common divisor from 300 to 4,000!