I don't think of it as an infra problem, but as an early manifestation of effort that would arise later on, anyway: long-term maintenance of that new language. You need people who know the language to integrate it well with the rest of the codebase, people who can perform maintenance on language-related tasks, people who can train other people on this language, ... These are all problems you'd have later on, but are usually handwaved away as trivial.

Throughout my career nearly every single company I've worked in had That One Codebase written by That One Brilliant Programmer in That One Weird Language that no-one maintains because the original author since left, the language turns out to be dead and because it's extremely expensive to hire or train more people to grok that language just for this project.