Downside is that, as a company paying someone to accomplish a task, I'd want to give a different task to each applicant, so that if they do a good job, it provides value to the company.

And this is a problem because rarely are two programming problems exactly as difficult as each other, so the test isn't normalized. Interviewing is hard enough when you keep all the variables that you can control the same.