2. Code reviews and security reviews happen quicker and produce more findings.
I would think that (m)any team(s) using AI might also be seeing a higher rate of finding and fixing issues.
Even the Linux Kernel (I'd say Windows and Apple too) are seeing the same phenomenon.
Linux Kernel: https://lore.kernel.org/all/CAHk-=wi4zC+Ze8e+p3tMv8TtG_80Kzs...
The idea that software has gotten so complex that a machine can evaluate code paths better than a human, seems to bristle the fur of many. Some people didn't think we would see the day where that comparative human limitation was laid bare in simpler tasks than they expected. I believe older developers are less likely to be offended, having to deal with this as a matter of course (as the mind declines).
And either way, what, we are going to keep this line going for another 5 years? Aren't you bored?
The citation was in support of the post above mine and was incidentally a link to a mailing list. I did not read the mailing list threads out of personal interest, admittedly. I think it's a particularly bad way to communicate (took 15 years for me to figure it out), so I avoid them.
> Like even in that linked thread, is personal offense like you lay out here
Taking it personally, is a concrete demonstration of what I described. The replies to my comment, are unsurprising.
Lol! What about fuzzers, linters, typecheckers and formal tooling? There’s plenty of machine code evaluators that people do use because it’s better than relying on human skills.
The issue is the actual report and the lack of information.
Pretty sure Chrome has been using all of those forever, along with some of the best security researchers in the world, yet AI (which is what GP really means by “machine” here) is finding way more bugs. I think GP’s point is that AI makes some people uncomfortable because it operates more like a human than a special purpose tool.
As for lack of information, I’m curious what you’d be interested in seeing. More information about the kinds of bugs it found perhaps?
https://cacm.acm.org/research/lessons-from-building-static-a...
This is the kind of report that you can reflect upon and learn from instead of feeling like you’ve just read a marketing piece.
I am very skeptical of these workflows, but I also use LLMs regularly. I specifically use them because they are better than me at sifting through massive amounts of complex information. They are also quite, uh, sketchy, for things that are significantly easier. A total mixed bag in my experience that ultimately is useful, and I will continue to use
Like having real difficulty with basic counting while being able to solve extreme math problems.
Being context machines, talking about what they’re definitely good at and definitely shit at, in broad terms, has been… difficult.