Besides, even if you assume conservation of particle number, it only has to be eventual conservation. Suppose I borrow some energy from "over here" temporarily and accidentally assemble them into a perfect brain for a second, then they can disperse again, if they like.
A bigger question is, if energy were conserved globally, where exactly did those handy particles come from in the first place anyway?