Can confirm, I worry about where I might have gone if it hadn't been for meditation pulling me out. It's impossible to put into words what it does for you, you're right, you just have to try it without any pre-conceived notion of what you want to get from it which for some, is too wild an idea.
Devil's advocate here. Lots people say the same thing about religion. In fact, you could almost replace meditation with religion in your comment. Quite illuminating.
And it's probably true for religion too. That doesn't constitute a counterargument. The negative aspects of religion (for example, you can argue that religious faith is exclusionary and confusing, and so on) don't carry over to meditation just because they have something in common.