https://www.chrisstucchio.com/blog/2014/why_xkcd_style_graph...
I see the same things in fonts that are supposed to look "hand-drawn", and CG renders of realistic scenes - it looks "imperfect" but the way the "imperfection" is itself perfect is what stands out. A little randomness goes a long way to avoiding that.
I use XKCD stuff for WAG's, not well-supported data.