Swipe a notification to the side a little bit, and you get a gear icon. Tap that, and you can set that app's max allowed notification importance. You can block all notifications, relegate them to only being visible when you pull down the notification bar, or allow normal icons but not "peeking" notifications.