I know someone who has a family who stated a side business in their spare time in their 30's.

It started extremely modestly using just some spare time, and I think a lot of people have at least some spare time. The average person spends, what, 4 hours a day watching TV? If OP has an hour a day they can make at least some progress.

This person also uses freelancers to help achieve chunks of work where they aren't experts. A modest amount of savings can pay for some of these deliverables.

I know someone else who sells baked goods at farmers markets. They put some time into baking and posting to social media but it's a side gig. Another person I know got together with a group to help share the effort and they're working on a hummus business (again, selling at farmers markets). These are all people with families and day jobs.

Instead of that fatalist do-nothing advice of "wait until you're an empty nester," I'd say "start small." Don't expect to be a massively scaling SaaS thing. Don't expect to replace your income, certainly not within ~5 years. Don't try to build some wildly complicated product. Budget your time, don't neglect your personal life. Focus on solving a simple pain point for a specific type of customer.

The other thing I'd warn OP about: don't start a business for the wrong reason. The wrong reason is dreaming of a lottery ticket out of wage slavery. The right reason is that you've identified an area that you think has potential to help someone else achieve a positive outcome, especially an area where you have some kind of expertise.