It won’t get me DOM level access but I can at least move a lot of my code there. I’m also quietly hopeful that canvas based rendering can make some huge improvements in the next few years so it doesn’t feel like Flash 2.0 but I’m ready to at least start thinking about letting go of the DOM as the thing I have to care about.
Until then I’m having a good time with Lit (lit.dev) for building web apps that need to be super snappy and “web feeling” which is still basically every customer facing thing.
But in a dream scenario I would way rather be writing apps in Flutter which was at least built from the ground up for building complex user interfaces in a sensible way, but that whole ecosystem is still in some very early days on the web and isn’t a good choice right now for most things, hoping that changes in a few years as they also seem to be targeting the WASM + Canvas path and the web as a platform isn’t there on that yet and neither are they.
Secondly, there is no way for automated agents to extract content from your user interface. This includes search engines, browser extensions, the browser itself, or your end users. I think that goes against what the web is, and I hope other devs agree. The mutability of HTML (and thus the DOM that represents it at runtime) is a strength, not a weakness
I apologise if I get some minor details wrong here as I am doing this on a phone and recalling this from memory because I don’t have the time to grab the sources right now.
However… the short version of the plan to solve this that I seem to recall to this:
The Flutter team specifically seemed to indicate that they are already used to operating in non DOM environments where they have to support accessibility across Android, iOS, MacOS, Linux and Windows that doing it on web actually isn’t that big a deal as it first seems.
They already have all of the code in place that builds a full tree (like the DOM) which does a complete mapping between every element (widgets in Flutter lingo) on the canvas to their respective bits of accessibility info. They then just take that tree and hook it up to the respective accessibility APIs that each platform exposes.
At no point have they indicated that this looked like it was going to be a serious roadblock or challenge for them. I believe them and they have a history of this approach working elsewhere.
There is just too much money riding on this investment for them not to get accessibility 100% correct in a web native way.
From there if you are able to expose everything through accessibility APIs then you presumably also have everything you need as a search engine or an adblocker to actively read and modify that canvas. At this point the entire argument that it’s just an opaque set of pixels seems to vanish for me.
This is also AFAIK already a solved problem for them in other products where they are using canvas based rendering such as Google Earth and Google Docs.
I don’t have the details beyond that right now sorry but that passes the sniff test for me at least.
