Once you are inserting models into non-creative (and software development is creative work) it is imperative to start creating your own small evals. Not just to check how different models perform in that equation, but to catch model drift over time.
1,550 karma · joined June 11, 2010
Once you are inserting models into non-creative (and software development is creative work) it is imperative to start creating your own small evals. Not just to check how different models perform in that equation, but to catch model drift over time.
I anticipate that the harness (and memory) will evolve far more outside of engineering. Muse/Dots are an example of that.
I've been writing about this thesis for the past year or so and here is a bit of a summary of where I think it is going: https://zero2data.substack.com/p/the-future-of-consumer-ai
Also, as tech companies grow, a significant portion of the dev body count is for internal tooling. I was always flabbergasted at Twitter's headcount growth back in the day when it seemed like the only innovation increasing the number of characters in a tweet.
Right now we are in a weird spot where a lot of people are flailing around inside corporate America trying to figure out how to leverage AI. Few have enough vision to see what the companies of the future will look like and work towards that.
1. There is one-time work (development, personal assistant) where the input and output are net new. There is also back office work (billing, FP&A, questions about data) that is quite repetitive--working within existing datasets.
2. Back office work exists within SaaS products all of which are including AI within their platforms. Those SaaS products are often integrated with each other.
3. Business logic needs to be transferred between systems in order for each SaaS AI to be accurate. (Asking a Salesforce AI and a Xero AI "What is our revenue?" may get you different answers if either system has custom logic that defines any input to that question differently.)
Based on this, I believe that the optimal path forward for back office work is:
1. Centralize agents (business logic) across an organization
2. Decouple agents from orchestration
Agents that exist at the end of an API call--which can be from a SaaS, Airflow, vibe-coded app, or model harness--are a better answer than platform specific AI. They provide a centralized repository for business logic while reducing vendor reliance (and lock in). Additionally, you get better observability and auditabilty.
Sleeper Service is the result of this thinking--an enterprise AI framework.
Larger write up of my thought process behind Sleeper Service here: https://zero2data.substack.com/p/agents-as-a-service
https://www.startuplevelup.com/corporations-as-an-api/
What I think is missing is the lineage of an agent's identity. A public key maybe?
When most people are consuming _their_ news (as opposed to _the_ news) in the form of a thumbnail in an algorithmic social media feed, those copy errors matter less.
Honest question. There is a line in the sand somewhere apparently.
Train ride videos on YouTube scratch some of the same itch.
I have the Smokey Mountain which is amazing at holding heat but a total PIA to clean. Have one of their gas grills as well on a built in line, and it is fairly high quality also.
Nothing tops the kettle though.
I am not anticipating people self identifying slop.
A feature I would like on LinkedIn would be a 100% verified human content flag for users. LinkedIn can then do the scans and flip that to false for any users it catches posting AI assisted content. Let me filter that out of the feed.
If we don’t do something to maintain some standard of discourse, we lose intellectually and as a piece of our humanity.
LinkedIn is the only social media I use and it is on thin ice.
https://www.mcsweeneys.net/articles/taylor-swift-a-socratic-...
Maybe my eyes are just getting worse...
https://www.youtube.com/live/hm9iAviOZ20?is=zkOMcU_8Sck0LOer
Ideal Partner: 35-55 | F | NYC/St. Louis | Ambitious, academic, or artistic -- someone who is intrinsically driven to learn or create
It was interesting to see where the approaches were similar and where they diverged.
I would like to remove the AI-generated content and any low-effort content from my LinkedIn feed. There can be some great stuff in there but a lot of it isn't worth my attention. Probably going to explore some browser plugins with that because the algorithm doesn't seem to do the best job giving me content that is interesting to me. It may be giving me topics I enjoy reading about now (AI) but not the level of "thought leadership" I want. Substack is better for that these days.
I write both places because I enjoy writing.
I think gp may want to know if a <person> has an interesting idea rather than <person + llm>.
I had thought this would be combined with OpenAI launching a set top box where you could talk to an AI avatar. Disney IP could have been skins to sell people for their AIs.
And also because my kids can spend tens of dollars in minutes on it.
Wrote a bit more here but that is the gist: https://zero2data.substack.com/p/trusted-prompts
I also recommend Founding Sales and think it would be worth the OP skimming.
Also, search for Steli Efti (founder of a CRM called Close) who has some great content for outbound sales. I thought he did a session for Y Combinator's Startup School but didn't just find it. But he has lots of great content and a bit of a hustle mentality.