It is actually harder to learn on your own if you don't have a set curriculum (that is, a dependency graph on academic subjects) and a guide to finding good materials (textbooks and such). Emphasis on the dependency graph: without one, it's far too easy to dive into something, get well over your head, and wind up more confused and frustrated than enlightened. This especially happens in the kind of deep, difficult subjects where you need the guidance of a good textbook or professor more, because not every teacher in the field actually knows the particular subfield.