The point of the Boltzmann brain is that it's spontaneous (which is also what makes it impossible). If a billion years of slowly ratcheting up complexity counts as a "random fluctuation", then every brain is a Boltzmann brain.