Whenever I see these “researchers get LLMs to succeed at X complex task” I wonder why I can’t even get it to do something basic in the real world (at my job). The gap seems vast
Yesterday, for example, I had some ts React class-based components I'd written which I wanted to convert to functional components. Select all, "convert these class components to functional". Done. Took a couple minutes to do this to a handful of components. A little manual cleanup after. This would have taken a lot longer manually.