195 karma · joined April 4, 2023
Some teams are using Claude or similar models in GitHub Actions, which automatically review PRs. The rules are basically natural language encoded in a YAML file that's committed in the codebase. Pretty lightweight to get started.
Other teams upgrade to dedicated tools like cubic. We have a feature where you can encode your rules either in our UI, or we're releasing a feature where you can write them directly in your codebase. We'll check them on every PR and leave comments when something violates a constraint.
The in-codebase approach is nice because the rules live next to the code they're protecting, so they evolve naturally as your system changes.
If you're curious, you can check it out here: https://cubic.dev
Happy to answer any questions about what we've seen working (or not working) across different teams.
That said, even with automated review, a 9000 line PR is still a hard reject. The real issue is that the submitter probably doesn't understand the code either. Ask them to walk you through it or break it down into smaller pieces. If they can't, that tells you everything.
The asymmetry is brutal though. Takes an hour to generate 9000 lines, takes days to review it properly. We need better tooling to handle this imbalance.
(Biased take: I'm building cubic.dev to help with this exact problem. Teams like n8n and Resend use it to catch issues automatically so reviewers can focus on what matters. But the human review is still essential.)
The good news with mrge is that it works just like any other AI code reviewer out there (CodeRabbit, Copilot for PRs, etc.). All AI-generated review comments sync directly back to GitHub, and interacting with the platform itself is entirely optional. In fact, several people in this thread mentioned they switched from Copilot or CodeRabbit because they found mrge's reviews more accurate.
If you prefer, you never need to leave GitHub at all.
If you're interested in Stack PRs, you should definitely check them out on Mrge. By the way, we natively support them (in beta atm): https://docs.mrge.io/ai-review/overview
We've heard from users who've tried both that our AI reviewer tends to catch more meaningful issues with less noise, that's really something you should try for yourself and find out! (The great thing is that it's really easy to start using)
Beyond the AI agent itself (which is somewhat similar to Copilot), our biggest differentiation comes from the human review experience we've built. Our goal was to create a Linear-like review workflow designed to help human reviewers understand and merge code faster.
If that's something your team might need, I'd love to chat more and keep you posted as we explore this!
For applying code changes with one-click: we keep suggestions deliberately conservative (usually obvious one-line fixes like typos) precisely to minimize risks of breaking things. Of course, you should confirm suggestions first.
Regarding AI reviewers being "too nice" and missing obvious mistakes—yes, that's a common issue and not easy to solve! We've approached it partly via prompt-tuning, and partly by equipping the AI with additional tools to better spot genuine mistakes without nitpicking unnecessarily. Lastly, we've added functionality allowing human reviewers to give immediate feedback directly to the AI—so it can continuously learn to pay attention to what's important to your team.
We've heard from users who've tried both that our AI reviewer tends to catch more meaningful issues with less noise, that's really something you should try for yourself and find out! (The great thing is that it's really easy to start using)
Beyond the AI agent itself (which is somewhat similar to CodeRabbit), our biggest differentiation comes from the human review experience we've built. Our goal was to create a Linear-like review workflow designed to help human reviewers understand and merge code faster.
I don't remember adding that feature so it might be a bug
And I absolutely love your idea of having multiple AI models review PRs simultaneously. Benchmarking LLMs can be notoriously tricky, so a "wisdom of the crowds" approach across a large user base could genuinely help identify which models perform best for specific codebases or even languages. We could even imagine certain models emerging as specialists for particular types of issues.
Really appreciate these suggestions!
Appreciate the feedback around security as well; protecting against supply-chain attacks is definitely top of mind for us as we build this out.
In the meantime, good luck with that hairy review—hope it goes smoothly! If you're open to it, I'd love to reach out directly once GitLab support is ready.