I looked it up. It was Einstein. :)
Or at least, it's attributed to Einstein. It's a common enough sentiment, though, I'm sure I've independently said similar things myself.
On the other hand, it can easily be taken too far. Sometimes a random factoid isn't just a random factoid, it's part of the scaffolding on which you hang your knowledge. If you have to go to your periodic table to look up things like "Carbon" and "Hydrogen" you're never gonna have an intuitive understanding of chemistry.