In those circumstances I tend to opt for inverted pyramid writing, in the hopes that the skimmers will get the gist and folks who really care for the details will get what they need.
But it's a tough balance, which is why it's critical to a) know your audience and b) have a clear understanding of what you're expecting when you're communicating. Are you sharing information? Communicating a decision? Seeking feedback or approval? All of those may require flexing into a different style to be most effective, particularly when taking audience into account.
And definitely check out the book "Style: Towards Clarity and Grace" by Joseph Williams and Gregory Colomb (there are several editions of this; but any earlier edition would do; I have the 10th edition). I wrote a bit more about it here[1] in the past.
One part of that is about redundancy and verbosity: with some effort it's usually possible to say the same thing with a lot less text.
The other part, as you say, is about knowing your audience: an executive didn't read your massive email because they genuinely don't need to. If you can pick out the few bits of information that they actually need to know then that's better for everyone.
As a sibling to your comment says, I have often unashamedly included an "executive summary". They seem to appreciate it. I also rather frequently pull them off the email chain when I'm diving into technical details, unless they explicitly ask to stay up-to-date on the entire chain. Then, if I want to re-update them, I'll add them back in, so they get the whole chain in one shot if they really want to dig and I'm not "hiding" anything, but they still get their summary.
Upshot, don't be scared to put "executive summaries" in. It's not sarcasm and it's not a joke.
An interesting thing I read about writing recently is the enormous difference between writing as if you are speaking to yourself, as juveniles typically do in school when they are asked to write a 300-word essay, and writing with an audience and goal in mind, which is everything in business writing.
You last paragraph succinctly states the technique.
You can't just keep piling lines upon lines, at some point you have to stop, re-read everything you've wrote, refactor it, make it simpler, more concise, remove the outdated text/code, etc. etc.
Documentation should always have one global "root" document, which contains links to everything else, and the documentation graph should be regularly reviewed and pruned.
I personally enjoy writing design documents and feel that they help me organize and systematize my thinking in a way that few other tools can. My coworkers don't share this view, however, so I usually end up writing for an audience of one.
