The PR description should contain the context that made a change necessary, which is something that can not be contained in the change itself and thus can't reliably be inferred from it.
And if I request a PR review, I trust you, as my peer, to understand the problem my change is trying to solve and to judge whether it succeeds in doing so. If I wanted AI suggestions for it I can skip the middleman.
I guess the "chat with the codebase" feature could be useful, but the auto-review feature just gives me very uncanny vibes. If someone would "review" my code by blasting it through an LLM I'd be ticked off.
For people who already know the code well, it can catch things you didn't think about, and that's kinda nice. I suspect it has cost me/my team more time on wild goose chases than it has saved on real findings, but it might still be worth it.
For others, I've seen it constantly mislead people, and convince them to make changes that look reasonable but lead to the PR being incorrect. Over and over and over. And the "look reasonable" part is particularly dangerous during code reviews - reviewers are less careful about things that look reasonable, that's a relatively normal and unavoidable thing for humans even though it's incorrect.
When it's exposed to everyone, there's a pressure to follow and trust it, like existing linters. Private use is less bad, but still risky due to the plausible-change-review-negligence issue.
---
Also PR descriptions in particular are borderline useless when written from an LLM, because they just summarize the changes (or mislead summary-readers about the change, when incorrect). So you're just littering your commit history with the equivalent of auto-generated blogspam. No thanks.
I am Tal, one of the developers of this extension.
See our FAQ regarding your concerns about the PR description, and the review process: https://pr-agent-docs.codium.ai/faq/#answer1
In general, PR-Agent is designed to assist, not replace, human reviewers.
Many of the flows and processes we deployed are aimed at meeting this goal. We try not to overclaim, or overpromise. PR-Agent should not be the sole reviewer of a PR. We believe it can definitely enhance and improve the reviewing process.
> And if I request a PR review, I trust you, as my peer, to understand the problem my change is trying to solve and to judge whether it succeeds in doing so. If I wanted AI suggestions for it I can skip the middleman.
Are you talking about keeping meaningful processes around? Because companies are trying to get rid of humans and people altogether. They don't really know why PR reviews exist - only that reviews can take time - and they don't want to spend time. But can't remove reviews anyway - so just paper over it.
I'd never trust an unrevised version though.
As is, getting in revised LLMs responses seems like it'll be Gen Alpha's "let me Google that for you" swipe. Except at least Googling means you still need to put in leg work.
My last company had codium on our internal codebases for PR review/summary. I found it neat, but I never saw it make any suggestions that surpassed linting. But as a lazy dev who didn't want to rewrite all the details I put in my commit messages when making a PR, I felt it did a pretty good job summarizing.
Funny story: One time I jokingly asked it to rate the skill level of the programmer who wrote a certain PR (which was created by me). It was very complimentary of my skills, which I shared in the humor chat about how it was a very smart AI. The very next day I was laid off (unrelated, it was along with most of my team).
What I love with these tools, I'm getting productive. Although, I'd still like to do the review myself.
