I believe that. I'm sure that I could have greatly reduced the latency, but tuning ffmpeg is not for the faint of heart, and my heart wasn't really into it.
Anyway, HLS has latency, just by definition. The "H" stands for "HTTP" (a synchronous protocol, based on TCP). RT[S]P uses UDP or RDT, and is isochronous.