Postfix supports a selection of block lists for spam, and I get no spam at all - although unfortunately my gf sends me email from a Yahoo account, and certain Y! servers get blocked, so that mail bounces.
She doesn't do it often enough for me to spend more time getting whitelisting to work. (Currently it doesn't - I don't know why.)
The hard part is getting a working config file for postfix, but there are tuts and examples online.
Like a lot of older FOSS code, postfix is basically an insane collection of every possible switch for every possible feature, dumped with no particular thought or care into a single config file, and written up - ditto - in a single help file.
You only need about 10% of it, but you won't know which 10% until you try.
The server gets regular hack/relay attacks from all over, but those get killed by Fail2Ban. I'm not exactly a high profile target, but unlike an old WP site I used to run - it was hacked in weeks - the servers seem to have survived for more than five years now.
