What fantasy world is everyone living in that they love their jobs and derive pleasure from it? What 1%'s of gardeners are we talking about here? I assure you 99% of them would rather not be gardening.
It's very likely if you are doing a physically laborious job you are making very little money, struggling to feed your family, give your kids the spoils they want, your wife is mad that you can't take her on vacation etc. etc.