>What Shiny does to get around this is natively 'evaluate' the plots on the backend, creating a rasterized PNG file

I don't think that's the case for D3 charts, because that would kill the interactivity that is so great about D3. I use RCharts to inject D3 into my Shiny applications, and have ran into into performance issues with just a couple hundred data points. I think this is because all heavy lifting is done by the client (browser), not the server.