Oh man, that was another nightmare with Books. I tried keeping syncing on for a year, but ultimately switched back to a local library.
It worked 99% of the time. But then every few weeks it would overwrite my newer-annotated version of a PDF with an older one, and I'd lose chapters' worth (in one case an entire books' worth) of highlights. It was infuriating.
It had to be some kind of iDrive sync bug. It seems insane that type of stuff passes QA, I can't even imagine.