That modularity is nice in theory, but Dojo gives me 170kb of compressed javascript simply for selecting the fileuploader widget. I saw the same for Google Closure Library by the way - not saying I could write an all purpose js library which would score better.
That's one of the major things that 1.7 fixes and 1.8 will fix even further... smaller modules and finer-grained dependencies so you aren't pulling in nearly as much base code for each feature.