Seeking the container is usually much faster than decoding and then throwing away what you don't need, but it has fatal flaw: most videos use p-frames and thus require you to decode the frames before it.
So, say you want to skip to 60 seconds in. The solution is to do "-ss 50 -i input.mkv -ss 10", which is fast and should get the keyframes you need.