I was reading an interview with Andy Matuschak[1].
> One of the things that I think is kind of weird about this memory system stuff, or like memory champions, or something like that is “Oh, if you do these things, will you start to forget other normal human stuff?” And what's weird is, no. I've been doing this memory system stuff for years and I just know more stuff now. This is aligned with the experimental literature, which seems to suggest that, there's probably upper bounds but we're not close to them. Some of these memory champions have memorized maybe two orders of magnitude more things than I have practiced. Certainly people who are multi-lingual have really, really absurd numbers of things memorized. So there isn't a resource management argument.
The notion that stuffing your brain with trivia will be damaging is merely an unwarranted fear.