Blimey, it was a decade or so ago. :p As they hit teen years just a normal PC+Internet was enough. From memory...
Mid spec, but old, games machine (mum or dad's old one after an upgrade).
Spent a bunch of time locking down what was in windows, what damage they could do (like deleting icons, menu items, random important things). Weekly restore from FreeBSD box. Uninstalled or removed visibility of pretty much anything that could do damage. :)
Mostly it was things that came up as they happened, and experience supporting non-tech friends and relatives - causing chaos uninstalling the wrong things, installing kazaa or whatever.
s/w wise I found a Logo (turtle graphics), Basic, couple of other languages, Amiga-like Music tracker, couple of graphics things to complement MSPaint, MAME with some classic games, a few infocom adventures and a couple of newer games for the Z interpreter (Infocom games not as popular as I thought they might be), a crossword puzzle and a few other logic/tile arrangement type puzzles.
Overall it was a great success - we didn't really need to teach anything.