But for conversation, you can’t speak Mandarin without using proper tones because you simply won’t be understood.
You can be perfectly understood if you don't speak broadcast Chinese. There are plenty of heavy accents to deal with anyways. Like Beijing 儿化 or the inability of southerners to pronounce sh very differently from s.