> Instead of us programming the AIs by feeding it lots of explicit hand-crafted rules/instructions, we're feeding the things with plain data instead, and the resulting behavior is much more black-box, less predictable and less controllable than anticipated.

I dunno, we do feed them lots of explicit hand-crafted rules/instructions, it's just that does don't go into the training process, but instead goes into the "system"/"developer" prompts, which is effectively the way you "program" the LLMs.

So you start out with nothing, adjust the weights based on the datasets until you reach something that allows you to "program" them via the system/developer prompts, which considering what's happening behind the scenes, is more controllable than expected.