They can infer that one IPV6 address matches to exactly one device. (reverse is not true, one device may have multiple addresses per privacy extensions)
Once device is identified all its past traffic is discernible.
Changing addresses means identification needs to be done again but once done it can be associated with past addresses and again, all its history is visible.
Identification might just mean querying a data broker with HTTP headers.
NAT does not have this issue.