One more advantage of keeping it together can be easier development cycle. IDE features like autocompletion and building would be faster if artifacts can be cached.
My Vim is indeed magic. I start typing in a name and it autocompletes, then adds includes for whatever the package the thing I just used is in. I also can't imagine going back to not having code search, with its turning every identifier into a link.