Granted, it isn't that physically demanding. The biggest amount of labour I find is repairing/modifying equipment, and that comes with interesting problem solving challenges, which my programmer mindset tends to enjoy.
Still, I can see why gardening would be seen as appealing.
> your wife is mad that you can't take her on vacation
Why would anyone get mad about that? My wife is a grown adult, she can pay for her own vacation. Farming is my vacation.