They're using SVG because support for these non-standard punctuation varies across browsers and OS's, so its safest to use an image.

Many sites presenting informational content about unicode content do the same, as they cannot presume the user will see the unicode symbols rendered natively in their browser.