Attention Is the New Big-O: A Systems Design Approach to Prompt Engineering (opens in new tab)

Yes: structure beats vibes. Primacy/recency bias is real. Treating prompts as engineering artifacts is empirically helpful.

But:

- “Reads all tokens at once.” Not quite. Decoder LLMs do a parallel prefill over the prompt, then sequential token-by-token decode with causal masks. That nuance is why primacy/recency and KV-cache behavior matter, and why instruction position can swing results.

- Embeddings & “labels.” Embeddings are learned via self-supervised next-token prediction, not from a labeled thesaurus. “Feline ≈ cat” emerges statistically, not by annotation.

- "Structure >> content". Content is what actually matters. “Well-scaffolded wrong spec” will give you confidently wrong output.

- Personas are mostly style. Yes, users like words typed in their style better, but it'll actually hide certain information that a "senior software engineer" might not know.

I don't really get the Big-O analogy thing, either. Models are constantly exposing and shifting how they direct attention, which is exactly the opposite of the durably true nature of algorithmic complexity. Memorizing how current models like their system prompts written is hardly the same thing.

Seems like an LLM should be able to judge a prompt, and collaboratively work with the user to improve it if necessary.

A lot of what I received as input was more like the first type of instruction, what I sent to the actual development team was closer to the second.

For example, pass an LLM a JSON structure with keys but no values and it tends to do a much better job populating the values than trying to fully generate complex data from a prompt alone. Then you can take that populated JSON to do something even more complex in a second prompt.

I found the “Big-O” analogy a bit strained & distracting, but still a good read.

Do you have preference n a more continual system like Claude Code for one big prompt or just trying to do one task and starting something new?

LLMs are random and unpredictable, the opposite of what real engineering is. We better start using terms like "invocations", "incantations", "spells" or "rain dance"/"rituals" to describe how to effectively "talk" to LLMs, because a science it most definitely isn't.

And yeah, taking the five seconds extra to do the bare minimum in structuring your communication will yield better results in literally any effort. Don't see why this concept deserves an article.

PS I am also extremely triggered from the idea of comparing Big-O, a scientific term and exact concept with well understood and predictable outcomes, with "prompt engineering" which is basically "my random thoughts and anecdotal biases of how to communicate better with one of the many similar but different fancy autocompletes with randomness built in".

In addition, for the technical aspect to make sense, a more effective article would place the points should be shown alongside evals. For example, if you're trying to make a point about where to put important context in the prompt, show a classic needle-in-the-haystack eval, or a jacobian matrix, alongside the results. Otherwise it's largely more prompt fluff.