This is not true. Finnish public health insurance is residency based: Every resident has it [1]. What you are probably thinking of, is the extra occupational private health insurance, that most Finnish employers provide for their employees (so that during work days, the employees don't need to wait in line in the public health services, but can get back to work sooner after seeing a private doctor).
[1] https://www.kela.fi/ulkomailta-suomeen-sairaanhoito-suomessa