If it can’t do something that basic and that common using a language and toolset with that much training data, then I’m pessimistic personally.
I’m yet to see Copilot be useful for any of my juniors when we pair, it gets in the way far more than it helps and it is ruining their deeper understanding, it seems.
I’ll continue trying to use these tools, but I swear you’re overselling their abilities even still.
This technique can reliably make any good LLM fluent in an API that it's never seen in its training data.
The LLMs you mention are first and foremost a “general knowledge” machine rather than a domain expert. In my opinion, Junior developers are the least likely to benefit from their use because they have neither the foundational understanding to know when the approach is wrong, nor the practical experience to correct any mistakes. An LLM can replace a junior dev because we expect the mistakes and potentially poor quality, but you don’t really want a junior developer doing code reviews for another junior developer before pushing code.
