For the edge cases that might need more like running a security audit, it would make more sense to submit that as a project proposal and have the expense approved.
For the edge cases that might need more like running a security audit, it would make more sense to submit that as a project proposal and have the expense approved.
It's easy to spend a lot when writing an entire service, large features, or doing research loops.
- freely experiment and understand the limitations of AI
- create things outside of what they are officially expected to
I’m not advocating for people who burn tokens for the sake of it. But I do think a single engineer has more potential for impact today than ever before and a high budget unlocks that.
It also isn’t _that_ much more when a fully loaded mid-level engineer is costing you $300k+
- Convert PPTX to TypeScript
- Allow editing of that TypeScript via an LLM interface
- Convert TypeScript to PPTX
The intent was to create slide deck templates via code. The first step was to convert our existing templates to this new format.
So I had it write a harness, gave them problem to Opus, and a couple thousand dollars later the import/export worked near perfectly.
At my company at least, I think our developers waste a ton of labor and money on getting LLMs to produce tools like this. Then they use tool a few times and ultimately forget about it or throw it away. At the end, they've spent a few hundred dollars on frontier tokens for the effort.
More time gets spent tweaking/iterating with the LLM than it would take to make the decks by hand. And sharing takes effort, so our devs mostly roll their own tools with minimal reuse. At the end there's no business value to show for it other than the slide decks that could have been made the old-fashioned way.
We already generated slide decks from a template. The decks generated were consistently missing data, incorrectly formatted, and required many manual edits.
This led to hiring contractors to fix up the decks before being sent internally before being presented to clients.
So, thousands of contractor hours every year going towards fixing up decks in additional to the engineering effort from ~5 bugs each month due to bugs in the deck or feature gaps.
> Then they use tool a few times and ultimately forget about it or throw it away.
This is a running production service
> More time gets spent tweaking/iterating with the LLM than it would take to make the decks by hand.
I think you misunderstood. The LLM produced deterministic deck templates in code that is then templated from a payload and the deck dynamically changes depending on what is in the payload. The user can then further modify the deck as they please since the deck is just code. Then they can export it to a PPTX and make any further adjustment in their favorite tool since it emits a "normal" PPTX.
See also, Matt Levine’s recurring joke about the crypto community constantly re-inventing/re-discovering the need for financial regulations that the larger finance industry have had for a while.
- Emits normal PPTX
- Can import any PPTX
- Representation can easily be edited by an LLM
- Handles most PPTX features -- rich text, transforms, charts/tables, images
- Can be templated
Things I tried:
- python-pptx, PptxGenJS
- Marp
- Google Slides + Google Sheets + Google AppScript
They could be tightening the purse strings further until they hit some magical ROI numbers.
You do not need much creativity for that. Just forget to put build artifacts or node_modules in your agent's ignore list and it will happily burn through the cap in thirty minutes