Prompts like "move the code relating to SQL query analysis into a new file", "look for opportunities to use pytest parametrize to remove duplication in that test", "rename method X to Y".
Early indications are that this is helping a lot with the problem where it's easy to churn out thousands of lines of code and not really have it stick in my head, even if I review every line of it.
Reviewing code and actively refactoring it is less tedious and more mentally engaging than reviewing code without changes.
If this was a human collaborator I'd be worried that I'm just creating busywork for them, but I don't care about busywork for LLMs!
The goal is to produce code that I understand and that I can remember just well enough that I get an updated mental model to help me productively make future decisions about the codebase.
I don't go along with their mitigations though.
In programming we have one tool for this: abstraction. Decomposition, pattern recognition, even data structures and algorithms are all down stream of abstraction. Collectively, we've never truly mastered abstraction, but it's what we have and we collectively wield it well enough that it's usually somewhat effective.
We are in dire need of a better abstraction.
We have chatbots in a sidebar that will just generate code for you or, more helpfully, answer your questions. We also have inline LLM code completion, which I've turned off completely because they're incredibly noisy.
What I want is something between those. My ideal use of LLMs while coding would be, i start writing a function and need to act on some data. I don't know what method to use, maybe I'm in an unfamiliar language/framework and don't know what my options are. I want the AI to explain what methods I can call to do X in this specific place, no more, no less. It would need to know what outcome I want, which would be hard to do without jumping out of the code and typing into the chat, but I basically want it to function like Intellisense on steroids. Something that doesn't break my focus.
Current LLMs are anti-flow. For me, that's poison.
I understand the rationale behind this, but can't help feeling that this is a downward spiral. The software industry has always been a hard place to build and sustain a career because of the pace of change. With these tools, the pressure to increase output is going to grow, jobs are going to be axed - so software devs need to work harder to stay relevant. Weren't these tools supposed to make our lives easier?!
As a dev with 40+ years behind me, I apply the same skills and tools to LLMs as I would to a small team of junior devs, except with a higher degree of rigor. I don’t feel that my skills are atrophying, in fact I’ve learned a bunch of new stuff in the process.
Depending on the work, I’m producing about 4x what I would have with a small team, and it is better code, better documented, more modularity, and less bugs than I was shipping with humans.
You have to keep a tight OODA loop, and write the code in specifications, protocols, interfaces, and implementation plans before the first executable line is put down. Iterate the code and docs together to minimize and document drift.
Treat a context window as a session. When it’s full, you start over with a new session, bring in the onboarding documentation, and go to work. I never cross context boundaries midstream, it’s always a disaster. 1m context is critical, I burn 150k just onboarding and orienting a new session.
Still shipping 4x at 1/10 the cost, and less bugs in the field (greenfield firmware development)
I like to do the opposite, asking the LLM to give me relevant follow-up documentation, like the actually docs, where I can read and understand things myself. Data structures, techniques, etc. I still like to read that from the authors, much easier and trustworthy to grasp.
I think this is how we should be reading code as well.
First understand the top level. Then the next level of detail and so on. I treat my understanding as graph of interconnected black boxes. If I don't understand a particular black box or a node in the graph. I click expand on it, grok the details and then collapse the node. Here's the grokking details of a particular sub-node also follows the same structure as understanding the root node. You don't need to understand everything from the get-go, expand your understanding on the need-to-know basis.
I'm really trying to do this too. The problem is it's *so easy* to let your standards slip, even for just a moment, and that piece of code suddenly becomes foreign.
I find more mental energy is spent on restraint than execution these days.
Second this. This is why zuckerberg is dying to spend as much as he can to make meta an AI company.
I've found that it's much more rewarding to use LLMs as an aid to deep work instead of a substitute for it, and it's even helped me feel more optimistic about my place in this field after a couple of days of getting used to the mental friction again.
Being conscious of what type of memory you're working in (or need to engage) may be the trick to building rhythm or flow, or whatever. Depending on the case the LLM may not even be necessary. Use something else.
The trap could be in trying to depend on and work with a model the same way we would work by ourselves, as the author describes, letting every type of memory unconsciously operate.
I hope the field moves out of the TUI with prompt + pull the lever paradigm soon‚ when it comes to agentic programming. And the Markdown paradigm too, tbh.
There hasn't been anything that really sticks yet for a shift to happen.
Now the question to the round: in your opinion, are LLMs ok to learn in this way? At least on the theoretical side of things?
This made me chuckle a bit.
There’s no point in fooling ourselves about our own skill retention if this is the case.
Thanks for this insight and validation
Imho most of professional coders trade their time for oblivion.
The problem is that you really don’t remember anything about the code. It is not your creation.
It’s like a monkey in front of a slot machine, just pulling the lever and waiting to see if it hits the jackpot.
At the end of the day, it remembers that it pulled the lever. And how many times it won :)
Agentic-based coding with /goal and multiple agents coding together is another level…
But the issue remain imho - if there is an error, who is going to repair it?
One thing that I look at is pushback rate: what percentage of the agent's proposals are rejected or critiqued? If it's below 5% I have found I have gotten too credulous and I am no longer closely following. Danger! If it's above 50%, I have clearly not given the agents sufficient context to perform the task and need to update my harness and instructions.
Who watches the watchers? I can imagine a guard dog process that halts the session to yell at the human if it detects complacency: if the human is providing too few tokens per minute of new context relevant to the task.
The experience will feel uncannily similar to AI generated code. So treat slop the same way. Give it a good, well tested API, and file an issue or PR when something breaks.
Can we be 10x more productive though? Or is it more like 1.25x? Is it AGI or is it more like an advanced compiler?
Unfortunately the world is betting on 10x when the reality on the ground feels more like 1.25x.
By which I mean, we should -- as software engineers -- be insisting on tools that put us in the driver's seat more.
Instead we're letting the agent drive. (I'm as guilty as this as anybody). But really we're letting Dario, Sam, Boris, etc. drive. And it should be clear from their public pronouncements and emissions that they don't have the best interests of our profession -- or the quality of software engineering generally -- in mind.
Yes, certainly, alter how you use the tools. But we need to fix the tools themselves.
