Typesetting something to a beforehand known format is radically different problem to a beforehand unknown format.
But TeX typesets to a beforehand known format (I assume you mean target document size) as much as HTML+CSS does (a bitmap canvas of <width> x infinity size): it's not optimized to reflow stuff live, but you can definitely change the page size before every run with a single command, and rendering should (mostly) reflow properly (this would be equivalent to resizing the window, or having differently sized/fine target like a cell phone).
It does not solve the problem of multiple page sizes any better than HTML+CSS, it just provides some nicer and more obvious tools for aligning stuff on the page/screen that could be more easily used to achieve such support.
Typesetting is a rendering a static copy a priori. You can expend a lot more time and resources rendering and reflowing the document. Rendering HTML+CSS, which can be dynamic, is a radically different problem.
I wouldn't be surprised if these posed very different constraints on the possibilities of the layout model
TeX's box model is in no way less optimizable than the HTML+CSS (if anything, it's the opposite due to it being much simpler). It has had a stable algorithm and implementation since 1983, and that implementation was never optimized for reflow like HTML + CSS has. TeX itself is also Turing-complete, which makes processing more expensive by definition, and that's mostly the reason for the slow performance, rather than the box model — and it is from 1983, after all. For some things, the reflow model is even used with TeX today (eg. table of contents generation usually requires at least two passes to "reflow" the output).
