Ooh, that's an interesting idea that I hadn't considered! My page is pretty computationally lightweight to start with (no 3rd party javascript) but if I'm ever in the mood for some unnecessary optimization that sounds like a fun one to experiment with.
Trade-off is slightly more data to send with each page, since you have to inline the SVG rather than having it cached after the first load. But it's a whole 2kb and could probably be smaller if I wanted to look into SVG optimization.