I wonder what the endgame here is, for our AIs to best reach that "I know kung-fu" capability. If we can evolve an optimized set of external skills, we can then distill them all into a model's weights, right? And if there is some benefit to being able to pick-and-match, would LoRA be the best approach? Or is there some real benefit to the free text approach?
