For instance, this skill of web development: https://raw.githubusercontent.com/vercel-labs/web-interface-...
That’s too much for a model to carry in its context while it’s trying to do actual work.
Far better is to give that skill.md to a model and have it produce several hundred lines of code with a shebang at the top. Now you haven’t got a skill, you’ve got a script. And it’s a script the model can run any time to check its work, without knowing what the script does, how, or why - it just sees the errors. Now all your principles of web dev can be checked across your codebase in a few hundred milliseconds while burning zero tokens.
TDD is codification too: codifying in executable form the precise way you want your logic to work. Enforce a 10ms timeout on every unit test and as a side effect your model won’t be able to introduce I/O or anything else that prevents parallel, randomized execution of your test suite. It’s awesome to be able to run ALL the tests hundreds of times per day.
Constantly checking your UI matches your design system? Have the model write a script that looks at your frontend codebase and refuses to let the model commit anything that doesn’t match the design system.
Codification is an insanely powerful thing to build into your mindset.
Maybe one should just search for advice from the last 20 years on how to make a human development team more effective, and do that stuff.
It’s funny how this advice has always been around, but we needed to invent this new kind of idiot savant developer to get the human developers to want to do it…
You can think of it literally as turning prompts that are non-deterministic into code. But it is also the idea of documenting all the things you learned while doing something. It’s what companies do when they invent process for humans (eg. “expense reports must be approved by your manager and finance before you get reimbursed”)
AI is a great to for exploring and experimenting with ways to do things- but once you know how you want it done, AI is just a liability because it might randomly decide to do things some other way, and it might do the thing differently tomorrow even though that’s not what you want.
Am I the only one who worries about agents creating malicious/unsafe code to execute?
I mainly work in Python, and I've been ensuring that all of my projects have a test suite which runs cleanly with "uv run pytest" - using a dev dependency group to ensure the right dependencies are installed.
This means I can run Claude Code against any of my repos and tell it "run 'uv run pytest', then implement ..." - which is a shortcut for having it use TDD and write tests for the code it's building, which is essential for having coding agents produce working code that they've tested before they commit.
Once this is working well I can drop ideas directly into the Claude app on my iPhone and get 80% of the implementation of the idea done by the time I get back to a laptop to finish it off.
I wrote a bit about "uv run pytest" and dependency groups here: https://til.simonwillison.net/uv/dependency-groups
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = [
# "requests<3",
# "rich",
# ]
# ///
import requests, rich
# ... script goes here`
I often have Claude Code build little HTML+JavaScript tools which are simple enough that I don't need a template ("Vanilla JavaScript, no dependencies or build script, test it with Playwright").
I've built a couple of Go apps which works great because Go has such strong default patterns and a full featured standard library, so again no need for a template.
MESA drivers are a godsend in Claude Code for web if working on non-web GUIs. It can take screenshots and otherwise interact with them.
I'm curious, for those of you who work like this, what level of subscription do you have?
Huh, this is...unexpected. TIL, indeed!
When I don't know what I want to do, I read existing code, think about it, and figure it out. Sometimes I'll sketch out ideas by writing code, then when I have something I like I'll get Claude to take my sketch as an example and having it go forward.
The big mistake I see people make is not knowing when to quit. Even with Opus 4.5 it still does weird things, and I've seen people end up arguing with Claude or trying to prompt engineer their way out of things when it would have been maybe 30 seconds of work to fix things manually. It's like people at shopping malls who spend 15 minutes driving in the parking lot to find a spot close to the door when they could have parked in the first spot they saw and walked to the door in less than a minute.
And as always, every line of code was written by me even if it wasn't written by me. I'm responsible for it, so I review all of it. If I wouldn't have written it on my own without AI assistance I don't commit it.
This is sage advice. I spent the better part of a day trying to steer Gemini into correcting an inconsistency when I likely could have solved it in under an hour. I think persevering with Gemini was due to a number of factors, including novelty, stubbornness, and (unfortunately) not knowing in detail what Gemini had written up to that point.
I eventually studied the resulting code, which ended up having a number of nested 'hacks' and required refactoring - more time wasted, but still much faster overall.
I think these calculations are very different now that we have these very good LLMs, but they aren't irrelevant.
Instead of having it write the code, I try to use it like a pair reviewer, critiquing as I go.
I ask it questions like "is it safe to pass null here", "can this function panic?", etc.
Or I'll ask it for opinions when I second guess my design choices. Sometimes I just want an authoritative answer to tell me my instincts are right.
So it becomes more like an extra smart IDE.
Actually writing code shouldn't be that mechanical. If it is, that may signify a lack of good abstractions. And some mechanical code is actually quite satisfying to write anyway.
> Sometimes I just want an authoritative answer to tell me my instincts are right.
You realize that LLM answers highly depend on how you frame your question?
Of course I do. I'm not a moron.
In these cases, I already know the most likely answer to my question. The LLM just helps me reduce my self doubt
Where I've automated more aggressively is everywhere around the code. My main challenge was running experiments repeatedly across different systems and keeping track of the various models I ran and their metrics, etc. I started using Skyportal.ai as an ops-side agent. For me, it's mostly: take the training code I just iterated on, automatically install and configure the system with the right ML stack, run experiments via prompt, and see my model metrics from there.
I’ll sometimes have it help read really long error messages as well.
I got it to help me fix a reported security vulnerability, but it was a long road and I had to constantly work to keep it from going off the rails and adding insane amounts of complexity and extra code. It likely would have been faster for me to read up on the specific vulnerability, take a walk, and come back to my desk to write something up.
For side projects? It's been a 10x+ multiplier.
1. The context window size to output accuracy problem seems like a genuinely hard problem
2. The vast majority of resources are being poured into driving down costs, not increasing output accuracy (and definitely not on gigantic context windows)
I will also add:
* I'm astounded at how terrible LLMs are at writing assembly in my experience. This seems like something they should excel at. So I'm genuinely confused on that piece.
The real issue is that I can't trust the AI generated code, or trust the AI to code-review for me. Some repeated issues I see:
- In my experience the AI doesn't integrate well with the code that there is already there: it often rewrites functionality and tend not to adhere to the project's conventions, but rather use what it is trained on.
- The AI often lacks depth into more complex issues. And because it doesn't see the broader implication of changes, it often doesn't write the tests that would cover them. Developers that wrote the PRs accept the AI tests without much investigation into the code-base. Since the changes passes the (also insufficient) tests, they send the PR to code-review.
- With AI I think (?) I'm more often the one careful deep diving into the project and re-designing the generated code in the code-review. In a way it's an indirect re-prompting.
I'm very happy with the increased PRs: they push the project forward, with great ideas of what to implement, and I'm very happy about AI increased productivity. Also, with AI developers are bolder in their contributions.
But this doesn't scale -- or I'll spend all my time code-reviewing :) I hope the AIs get better quickly.
Forgive me because this is a bit of a tangential rant on the second issue, but Gemini Pro 3 was absolutely heinous about this so I cancelled my sub. I'm completely puzzled what it's supposed to be good for.
To your third issue, you should maybe consider building a dataset from those interactions... you might be able to train a LoRA on them and use it as a first pass before you lift a finger to scroll through a PR.
I think a really big issue is that there is a lack of consistency in the use of AI for SWE. There are a lot of models and poorly designed agents/assistants with really unforgivable performance and people just blindly using them without caring about the outputs amounts to something that is kind of Denial-of-Service-y and I keep seeing this issue be raised over and over again.
At the risk of sounding elitist, the world might be a better place for project maintainers when the free money stops rolling into the frontier labs to offer anyone and everyone free use of the models...never give a baby powertools and so on.
AI coding tools are burning massive token budgets on boilerplate thousands of tokens just to render simple interfaces.
Consider the token cost of "Hello World":
- Tkinter: `import tkinter as tk; tk.Button(text="Hello").pack()`
- React: 500MB of node_modules, and dependencies
Right now context windows token limits are finite and costly. What do you think?
My prediction is that tooling that manage token and context efficiency will become essential.
function Hello() { return <button>Hello</buttton> }
The more code, the more surface area the LLM needs to cover before understanding or implementing correctly.
Right now the solution to expensive token limits is the most token-efficient technology. let's reframe it better. Was react made to help humans organize code better or machines?
Is the High Code-to-Functionality Ratio 3 lines that do real work > 50 lines of setup really necessary?
But my side projects which I kinda abandoned a long time ago are getting a second life and it is really fun just to direct the agent instead of slowly re-aquire all of the knowledge and waste time typing in all the stuff into the computer.
I wrote beads-skills for Claude that I'll release soon to enforce this process.
2026 will be the year of agent orchestration for those of us who are frustrated having 10 different agents to check on constantly.
gastown is cool but too opinionated.
I'm excited about this promising new project: https://github.com/jzila/canopy
We're writing an internal tool to help with planning, which most people don't think is a problem but I think is a serious problem. Most plans are either too long and/or you end up repeating yourself.
AI excels at finding the "seams," those spots where a feature connects to the underlying tech stack, and figuring out how the feature is really implemented. You might think just asking Claude or Cursor to grab a feature from a repo works, but in practice they often miss pieces because key code can be scattered in unexpected places. Our skills fix that by giving structured, complete guides so the AI ports it accurately. For example, if an e-commerce platform has payments built in and you need payments in your software, you can reference the exact implementation and adapt it reliably.
Looks like this: https://mastodon.social/@rcarmo/115937685095982965
SyncThing syncs their workspaces to my desktop/laptop, I make small adjustments (I don’t do stupid wasteful things like the Ralph approach, I prefer clear SPEC documents and TODO checklists, plus extensive testing and switching models for doing code audits on each other), my changes sync back, etc.
I’m considering calling it “million monkeys”, really.
Testing is no yet the main focus, so we haven't looked into automating that. But we will in the future. We have automated most of our documentation updates though. After releases or big merges we use askmanu to automatically update/create the docs. These are internal docs, but super useful for us and Claude.
Disclaimer: I'm the founder of askmanu
The first saves me days of work/month by sparing me endless paper pages of notes trying to figure out why things work in a certain way in legacy work codebases. The second spares me from having to dig too much in partially outdated or lacking documentation or having to melt my brain understanding the architecture of every different dependency.
So I just put major deps in my projects in a `_vendor` directory that contains the source code of the dependencies and if I have doubts LLMs dig into it and their test to shed light.
What I haven't seen anybody yet accomplish is produce quality software by having AI write them. I'm not saying they can't help here, but the bottleneck is still reviewing and as soon as you get sloppy, codebase quality goes south, and the product quality follows soon after.
It doesn't write the code for me, but I talk to it like it is a personal technical consultant on this product and it has been very helpful.
I have some complex large and strict compliance projects that the AI is a pair programmer but I make most of the decisions, and I have smaller projects that, despite great impact on the bottom line, can be entirely done unsupervised due to the low risk factor of "mistakes" and the easiness of correcting them after the fact they are caught by the AI as well.
The more time you spend making guidelines and guardrails, the more success the LLM has at acing your prompt. There I created a wizard to get it right from the beginning, simplifying and "guiding" you into thinking what you want to achieve.
/gh-issue [issue number]
/gh-pr [pr number]
Edit: replaced links to private github repo to pastebin.
The generators typically generates about 90% of the code I need to write a biz app. Leaving the most important code to me: the biz logic.
No AI. Just code that takes a (simple) declarative spec file and generates Typescript/C++/Java/... code.
I am also using AI's daily. However the code generators are still generating more productivity for me than AI's ever have.
True I spent a year making a platform for using the API.. but the results... are stupendous!! Very cheap and unlimited access and custom tooling, etc... to get large amounts done of anything you want to do with an LLM!
What would you make if you could make anything? Does it all just lose meaning?