I think that is the main obstacle to most people doing highly effective work and putting in long hours. You hear some call people who don't 'work hard' lazy, but my impression is that it's emotional capacity, and a lot of that comes from family.
I wonder if there is a correlation between prodigies and emotionally stable, healthy, present parents. It's hard to imagine children under a lot of stress - e.g., from abusive parents, highly unreliable parents (e.g., overwhelmed by addictions to drugs), emotionally unstable parents (e.g., narcissists), highly neglectful parents (e.g., who abandon their kids) ... - it's hard to imagine those kids doing what Tao did, regardless of their talent.