This is pretty much true, many people do. You can get your paperwork sorted, buy a car, a bank account, deal with the government and have a social life all in English. I have done.
But, all of the comments I'm seeing here are saying that while this is true, if you want to take the step forward into learning Finnish, it _is_ harder than other languages and that the resources around doing so as an adult are not as developed as in other languages. The fact that you don't strictly need to and that a lot of people have excellent English here anyway makes it harder, therefore active and sustained effort is needed on your part, you are not going to 'just pick it up'. If I had lived in France for as long as I have in Finland (not even a full year), then there is no question that my French would be at a much higher level than my Finnish is now.