It's possible for now but you lose all your history, settings, etc. And my Google Voice number has been primary number for a decade or so and thus has a lot of important stuff that I would like to keep.
I was able to migrate a Google Voice number and history from one @gmail.com to another @gmail.com - IIRC there was someone on Google support forums who I messaged with and after validating accounts they put it in some queue or something. IIRC a day or two later it was all there. Magic.