just the fact that it feels _unnatural_. I'm not saying it's wrong, just that personally I don't feel it's a good thing. I know it's tracking only the things you add to it, but still. I think my problem with it is that it feels messy.
My approach: git repo with a very simple setup script that creates symlinks to the dotfiles/dotdirs. It also manages some sensitive things like gpg/ssh keys.