For example:
// How: This filters records by MyType of J and if they have children
// Why: We want Jobs (J) when those Jobs have Children. Otherwise, we'll get JobGroups or Jobs without any Children we need to process below.
.Where(T => T.MyType == "J" && T.Child.Count() > 0);
Really? You don't think it's a matter of updating the prompt and expanding the context to include more of the code?
I don't know the exact method of how they generated these comments, but I think just focusing a prompt on a single method with the entire codebase in context would yield much better results.
* Past incidents
* Regulations
* What the code used to do, and why that was bad
The whole point of a really good comment is that you can't infer it just by looking at the code. That's why it's there at all!
AI isn't magic. If the "why" isn't in the code, there's no way for it to magically synthesize it.
Comments explaining "there's a check for this edge case because we saw a problem in production when client X did something, see JIRA-1234 for more details" are what you should be aspiring to. If a human doesn't write that down, the AI can't invent it.
And funnily enough, bad advertisement. How could a startup that focuses on code think that making a PR with 3,158 changed files is acceptable?
I just experimented a little bit, and telling chatgpt to assume a self-documenting approach and skipping comments whenever possible, gets a pretty minimalist output. Add on asking about what non-obvious aspects might be noteworthy based on other projects, and I bet chatgpt would be able to find a list of things to look for, see if it's applicable, and then rewrite in the appropriate format. Like if the function is about sorting, it could figure out that knowing if it's stable is an important marker, and figure that out. Something like that could be useful.
isUserAMonkey
Added in API level 8
public static boolean isUserAMonkey ()
Returns "true" if the user interface is currently being messed with by a monkey.
What is this code for. How would they even know its a monkey?!
I'm sure its a joke of sorts, or clever moniker if you want to be charitable, but unclear to me in any case
A product that scans your repo and identifies mis-named functions would be compelling, though.
It reflects back on them really well.
I've never used Spring Boot, but I have to give them credit for this. I don't know that I'd have similar restraint
I’d be surprised if their team wasn’t incredibly professional.
Things like a method called Foo that takes a Bar so the comment says (in 3 or 4 lines)
“Foo takes a Bar”
Wow! The future is here!
One is that (glancing through the comments) they're actually significantly more thorough than what is already there and they're at the level of thoroughness that a new user may actually want. Phil Webb is quite right; this is the level of detail someone new to the codebase could use, and if you could generate it dynamically on the fly as a "Help me understand what I'm looking at" tool that'd be really nice-to-have.
Second: I've definitely worked at places so tight-ass about code documentation that they do want "makeAppleRed(): Makes the apple red." Mostly because they're using doc engines that do a bad job of letting you know a function exists at all if it isn't documented. I have no doubt Codemaker is going to make its money on those places.
Consider this real example from the pull request:
/**
* Returns the plugin version property.
* @return the plugin version property
*/
@Input
public Property<String> getPluginVersion() {
return this.pluginVersion;
But contrast with this example: https://github.com/spring-projects/spring-boot/pull/39754/fi...
A function named `apply` with no documentation; you have to go up to the class to find the docs. And yes, it's not DRY to repeat the docs, but the fact the tool can pull relevant context into this location is helpful if it could be used as an on-the-fly doc generator (assuming, of course, it gives true output).
People (understandably) don't want to repeat themselves but they also can't predict what entrypoint a reader enters their class from. I've definitely bounced off APIs before because something like `apply` isn't doc'd even though it's doc'd in source twenty lines away (because I'm busy; I'm not scanning entire source files on the off chance that the thing I'm looking for is nearby).
Here's a sample of the crap you get, for 3000+ files, multiple comments per file, all this inane, that anyone with two brain cells could figure out just by reading the name of the function. If the project didn't already have these sort of comments, then adding them now is a choice that the maintainer of standards would have to make, not some random contributor.
The second point doesn't seem like a positive on the whole, except maybe for the company selling it, in the "there's good money to be made in being part of the problem" sense.
Edit: I noticed one comment that seemed like it added some useful context: For CheckAdditionalSpringConfigurationMetadata.Report::iterator it says "Each line represents a source code file and its associated problems." However, this turns out to be completely wrong, if I'm reading the code right, because a line could represent a filename, or a problem, or just be a blank line. The comment gives the appearance of being helpful while actually just adding confusion and wasting everyone's time.
No argument there (I don't miss those firm's doc standards). But you can make a lot of money being part of the problem in the business universe.
> The comment gives the appearance of being helpful while actually just adding confusion and wasting everyone's time.
That's definitely a concern; if the tool doesn't work it doesn't work, potential or no.
I think the first comment (https://github.com/spring-projects/spring-boot/pull/39754/fi...) looks helpful, although it is repeating info you can find in the class docs, the fact you have to go to the class docs to find it is an issue. A tool that could on--the-fly generate "Give me context for this function" (and give the right output) would be pretty useful for comprehending a codebase.
Always up-to-date and succinct auto-generated LLM-generated 'documentation' that describes a method in terms of who calls it (and perhaps why?) and the behaviour of the whole deep call graph inside the function seems like something that could be quite useful when looking at a method that doesn't have any human written documentation available, and isn't obvious... but it shouldn't masquerade as if it's actual API documentation that describes a durable contract.
If this synthesised analysis could flag incompatibilities between the contract described in API documentation and the actual code behaviour that could be useful (similar to what tools do today for nullable/non-nullable-annotated args/returns, except on fuzzier natural language descriptions of higher level behaviours). That seems like a much harder reasoning problem for the LLM to solve, though.
Note: I'm not talking about things that you know just by looking at the method itself, but things that are obvious when reading the code of the method only.
/** This will return the data */
Data getData() {...}
/** This will return true iff it's not empty */
boolean isValid() {return !empty();}
Edit: remember that this is library that other people will use, it's not an internal tool that only your team knows about.