Yeah, there is a lot of bullshit around psychedelics, and it's totally possible to take them and just have fun, but if you're interested in the way the mind works and how we piece together reality from our perceptions, there is a lot to work with there. I think where people go wrong is in thinking that drug experiences show you some truth that's hidden from you in normal life. It really just shows you how ill equipped your brain is to understand reality, or how ill-conceived the idea of 'reality' really is.
I think the idea of seeing some hidden truth, comes simply from seeing the world from some other perspective. I guess the problem people have with that is that people who take drugs talk like their epiphanies are only obtainable by doing drugs (and thus unique and rare). When instead drugs are more tools to find out things you personally didn't know.