Nobody wants to be the one coming to his boss with problems; nobody wants to be the only team member to go talk about things that probably bother him given that he never asked about them.
Then obviously he has no idea about the day-to-day work that most of us do, the difficulties we're facing, the lack of motivation of some members, the career or personal aspirations of them, the growing idea of leaving the team of others…
If you never ask, there's a lot you'll never know. It doesn't need to be weekly, but you need to reserve some time at least every few months at the bare minimum.