Ability to trivially use custom VM images was quite nice. The amount of CI time spent installing dependencies or copying a cache of installed stuff is nontrivial. Particularly for Windows the time difference is often very substantial. But even for plain Linux, there's no point in apt-get update && apt-get install the same set of things in every run (when using containers, cirrus could build them in-demand too, with little notational overhead).
Defaulting to throw-away-VMs for everything is also the right choice for something where the threat model includes attackers submitting patches/PRs. I'll never understand why folks were ok with just container separation for that (and often have no separation in runners).