Hacker News Reader

I find that AI models are very bad at doing performance work because they keep guessing how their changes affect things or not really understanding how the profiler results work, leading to them going in circles and taking forever. I have noticed this effect in surprisingly few lines of code (hundreds).

View on HN

One thing I've found that I've found super helpful for this is converting profiling results to Markdown and feeding it back into the agent in a loop. I've done it with a bit of manual orchestration, but it could probably be automated pretty well. Specifically, pprof-rs[0] and pprof-to-md[1] have worked pretty well for me, YMMV.

[0] https://github.com/tikv/pprof-rs

[1] https://github.com/platformatic/pprof-to-md

Yes but the problem is that the agent reads the profile and doesn't seem to really understand how to improve things. For example, it will see "cycles are spent in GC" and make up a bunch of reasons why that might be happening.

do you have a solution that still includes the model in the loop or do you just look at the profiler results yourself and then direct it what to do?

Well I basically cannot use it for performance decisions at all. I make all the choices and use it for simple things like "find me the top functions in this profile" or "refactor this code to test xyz prototype".

Reply on news.ycombinator.com