I'm fairly confident a coding specific model should be a lot smaller - 3b should be plenty if not 1b or less. As it stands, there are quite a few 7-13b model sizes that can predict natural language quite well. Code seems at its surface a much simpler language, strict grammars, etc so I wouldn't think it needs to be anywhere near as large as the nlp models. Right now people are retraining nlp models to work with code, but I think the best code helper models in the future will be trained primarily on code and maybe fine tuned on some language. I'm thinking less of a chat bot api and more of a giant leap in "intellisense" services.