Something in the direction of SVG is probably correct. I like the idea of having arbitrary paths available. One common annoyance with existing tools is that there is sometimes a lack of shapes and colors that express what I want. I find the same even with Google's rudimentary drawing tools.

It may be the case that these diagraming tools are aiming to be too high level. But where the middle ground lies I am not too sure. It is nice to express high level concepts like "Actors" or "Process" or "Class". And it is nice to have convenient syntax for things like UML sequence diagrams. I don't think it would be an equal exchange to give me rectangle and line drawing primitives since now I'd have to manage the connections by hand. And there would be no semantic meaning in such text formats, even if you managed to retain "readability".

But on the other hand ... to get the diagram solutions into a visual format I like I tend to lose the readability anyway. If you have several boxes with many connection - even reading the text becomes difficult. It is hard to keep the connections in your head once they get 2 or 3 hops deep. And you end up with 15 lines of connection spam, like "EntityA -> EntityB", "EntityC -> EntityB", "EntityA -> EntityD". And you end up trying to manage hierarchical relationships, fussing over what is the correct order to declare things, like manually doing your own dependency resolution. So maybe strict readability isn't really even a property I can expect to maintain.

I really hope it gets solved one day.