I found an abbreviated quote from the bit I’m thinking of in Bluebeard. Loses some of it, but gets his point across:
https://www.goodreads.com/quotes/277466-simply-moderate-gift...
But I am quite sure I saw similar sentiments at least one other place in his work, and I think a couple places—years and years ago I read most of his novels, plus most of the collected short fiction and short stories, but it’s all pretty fuzzy now.