1) Retention sucks in the current model of higher education.
I've been thinking about this one a lot lately. I've been doing a one year masters where I'm taking two courses a semester and doing research. The depth with which I am learning things is night and day compared to the depth with which I learned my undergrad material. During undergrad, I was drinking from a firehose and just trying not to drown. I would turn in unfinished problem sets, not having learned the material, and move on with my life. I would sleep through classes out of sheer exhaustion.
Now, with just two classes, I'm able to learn things almost well enough to teach them. So one way to improve retention is just to take things slower.
Another model comes to mind if we consider how people study at Cambridge, Oxford, etc. I have not experienced it myself, but according to students who went there as exchange students (and students from there who came here (here being MIT)) it's pretty different. Students here are overwhelmed with constant work. There, it is a lot more self paced, with a set of final examination at the end (someone please correct me if I am not doing it justice). So perhaps self-pacing and working smarter, not harder leads to more retention.
Do you know of any sources for retention statistics such as those you cited? Some of them don't match my experience (for example, I would say that I spent 8-10 hours a week in classes related to my major).
2) Courses need to happen in a logical sequence so that they can build on one another.
When I first read your post I thought you were suggesting that students should, by themselves, pick what to learn based on what they want to build, in lieu of being guided through a logical curriculum.
The point about scheduling gaps is interesting. Scheduling gaps happen because it's hard to satisfy the constraints of so many student and faculty schedules. If you could take courses on demand, that would fix things.
3) Results driven learning can be excellent for motivation and retention.
When one talks about results, there is a fundamental issue of time scale.
Courses that say things like "When you're done, you will have built an autonomous mobile robot" are great.
But there are many fundamental things to learn, over a long period of time, whose benefits
* you might not see for a long time * are broader than you could have ever imagined (and hence the benefit would seem artificially low to you)
If you as a student get to pick the desired result yourself all the time, you might be tempted to pick shorter-term results. This can be catastrophic to your education.
I believe in forcing people to learn fundamentals of their chosen field --- fundamentals whose power they might not appreciate until later. Learning fundamentals (that you might choose not to learn if you weren't forced to) is fruitful in powerful and unexpected ways.
Take pure math classes. You learn analysis. Then you learn measure theory. Then you learn measure-theoretic probability theory. Then you learn stochastic processes. All of a sudden, financial mathematics becomes easy to grasp. But so do a host of other things. Signal processing, computer vision, statistical mechanics, complex multiagent systems, epidemic modeling, control systems.
I suppose you could have started on this path because you wanted to learn financial mathematics. But it probably would have seemed way too complicated and difficult. But if someone says they want to be an applied mathematician (a much "broader" and more long term goal than just learning financial mathematics), then they'd better take a ton of pure math.