When a customer starts saying “we need to build X”, first ask what the actual problem is etc. It takes actual effort, and you need to speak their language (understand the domain).
But if you have a PM in the middle, now you just start playing telephone and I don’t believe that’s great for anyone involved.