Bitbake doesn't model all changes that affect packages, so after certain changes, some packages that should be rebuilt, aren't. It is especially prone to happen when changing Bitbake variables (example: MACHINE_FEATURES), and these are a quite common way to change things about the image being built.