Historically this exact same thing has happened, it was one of the bigger arguments against the abolition of child labour. "How will they grow up to be workers if they're not doing these jobs where they can learn the skills they'll need?"

The answer then was extending schooling, so that people (children at the time) could learn those skills without having their labour exploited. I would argue we should consider that today, extend mandatory free schooling. The economic purpose of education is that at the end of it the person should be able to have a job, removing entry level jobs doesn't change the economic purpose of education, so extend education until the person is able to have a job at the end of it again.

The social purpose of schooling is to make good members of society, and I don't think that cause would be significantly harmed by extending schooling in order for students to have learned enough to be more capable than an LLM in the job market.