Conner Ardman – I Tried Every AI Coding Assistant [video]

The factorial example is interesting because it shows how the programmer's own knowledge affects how to interpret the result of the tools.

Factorial overflows pretty quickly. I think 23! is where Javascript can't handle it correctly any more - far less than the 170 from CodiumAI, and long before stack overflow issues.

CodiumAI generated a correct test for 30!, but the presenter didn't run that test. In fact, CodiumAI was the only one to generate a test case that was high enough to exercise precision loss.

SourceGraph Code suggested using BigInts, but the presenter didn't think it was useful. One side effect is that n! is no longer O(n) with that switch, because multiplication is O(n^1.585) using the Karatsuba algorithm.

EDIT: Oh, of course - the 30! test will work as both values will be stored as 64-bit floats with the same value. The real test for no precision loss is that factorial(30) != 265252859812191058636308480000001 but none of tests handled that possibility.

Also, the test for n>=170 is really more an stack overflow prevention as the value will be infinity anyway.

i've been switching between WebStorm and VS Code to use cody and copilot together. i find that copilot does better with autocomplete and cody chat is better since i can choose between multiple LLMs like Mixtral 8x7B which is working better for my react project.