ericmcer

Current tool calling just parses llm output for flags that indicate a tool should be used.

How would you do that when output is raw audio buffers. You would need your conditionals to be like: if (audioDataArraySeemsLikeAnEmail(llmResponse)) {}

How would you feed db/knowledge queries back to the model? Do TTS and try to stuff context with the synthesized audio?

>> I'm wondering what the utility of this would be.

Exactly, it does feel next level compared to STT -> LLM -> TTS setups, but unless you can steer it effectively to do something like helping a customer do a complex task it is just a chatbot?

I can't even imagine what things like RAG, system prompts and other context setups would look like with this.

can a model like this be used to replicate a loved one’s voice? isn’t it a model trained on audio instead of text? So synthesizing another voice would involve rewriting the weights?

the current dynamic is still an engineer manually auditing and signing off on code.

why wouldn’t that work for other high trust industries (law, medicine, government, etc.)

The Neanderthal extinction always made me think that erasure of races in our modern times will follow a similar pattern. People think of the Native Americans as getting wiped out by disease/violence over a few years, but their ultimate end (and every other races end) probably will be similar to Neanderthals: slowly getting bred out of existence until finding someone with >2% DNA is a novelty. Same thing will probably happen to any smaller populations as globalization/interbreeding means massive populations like China/India will end up being the majority of the DNA.

when are they going to do something actually impactful though (like drive a real innovation in the physical world)? The most invested in areas are coding and image/asset generation and those both still require careful supervision and guidance to achieve anything usable.

The tools are crazy impressive but is this really the future we dreamed of? Cool now I sit on the computer and monitor n+ agents instead of doing it myself. Great it hacked Huggingface "accidentally" wow AGI post-work world is coming soon.

That is a great example, and kind of frightening how once the market decides to start pumping money into something our world morphs to meet the flow of cash.

AI is definitely impressive but are LLMs the true path to AGI? Is AGI the path to a better life/world? These tools are really impressive but it would be nice if they did something actually useful. Can a rogue OpenAI agent draw investor attention by curing a disease instead of hacking into the CIA?

displaced because the land has become so valuable you can no longer pay the tax on it and have to sell for a massive profit?

boo hoo? My generation has never even had access to affordable housing. Sorry you have to take your millions and live on a smaller property for your last few years.

This isn't talked about enough, whenever people mention declining fertility rates it is all about the economy and boomers and climate change dread.

If our current situation was that bad than we would never have made it as a species. We have been cranking out babies through way bleaker times than this.

One annoying part about all this is that there are basic abstractions (model, harness, classic networking terms) that would make all these behaviors quantifiable to a relatively competent software eng. Instead it feels like they generate jargon and new abstractions to give an air of mystique to everything they do.

The reality is they are just doing more complex model/harness/skills type setups but instead of just saying that you need to learn 20 new terms and parse through a bunch of hype/paranoia to drill down to what actually happened.

Anthropic published an article about changes over time in how long it took engineers to approve AI code. Basically the longer an engineer uses AI the faster they start approving it's code changes. I don't know if it is laziness or just how we are wired to behave but that is the nature of AI programming.

That has been a part of post mortems everywhere I work, there was never a name for it though. Just someone would say: “how do we prevent this in the future?” which usually spawns a few tasks that the person who caused the issue or others will pick up.

It might have more to do with legacy files than that being the best setup they could find.

They probably have tens of thousands? hundreds of thousands? of 3D files at this point, organizing, reusing, composing all those parts is necessary unless they want to manufacture everything from scratch every time. Migrating all that over to a new system wont be done until it is insanely painful to use the older system.

I always wondered if this was as simple as reforming the insurance companies, or if the entire healthcare industry needed to be reworked.

>> Medicare for All would prevent an estimated 68,000 unnecessary deaths a year.

Are there doctors and specialists with the necessary equipment just twiddling their thumbs, wanting to help people but the insurance companies refuse to pay so they just sit idle? Or is our healthcare limited by supply?

We might have to revisit a bunch of things like how doctors get trained/residency, what insurance should cover, public health around prevention, etc. rather than just saying "Give everyone everything".

The tooling for industrial farming is reliable on the scale of millions though, and the same for any other innovation that has replaced older ways of working. AI is in a weird place because it is not 100% reliable, and we do not know if it ever will be, but we also do not know if it will eventually good enough to where we are irrelevant.

If tractors plowed reliably 95% of the time, but unless carefully monitored the other 5% of the time they would blow a big hole in the ground, it would be difficult to abandon your shovel entirely.

Well yeah it conditions you to turn towards it anytime you hit a roadblock.

It is in a precarious state right now where it is too good not to leverage but will bite you if you get sloppy.

I still write things by hand sometimes because it is fun, but it does make you feel stupid when you spend 2 hours fixing something a 2 min prompt could have fixed.

I am wondering how this will effect other industries more. We are pretty flexible as software engineers because our skillset needs to constantly evolve. For a lawyer or radiologist or anyone who has been able to do the same thing every day for 20 years? What kind of reaction will they have when they see the ground shifting beneath them.

I totally agree with you, it is important for people to understand that a medical doctor does not really care if you can walk a mile without pain, but it is way too nuanced to track quality of life changes across a large population. We would need to change the insurance landscape and make it easier for other medical practitioners to get insurance money and let it naturally change.

If even a little bit of the insurance money could start funneling into things like physical therapy (actual physical therapy not the BS artists at big hospitals who print you out a sheet of generic exercises) people could see practitioners who actually care about you being functional.

I tore my achilles and saw a string of doctors, podiatrists, physical therapists, chiropractors, etc. before finally finding someone that doesn't take insurance but who had me running again within a few months. He was cheap compared to the amount my insurance was being billed for a chiropractor to try and sell me $500 custom insoles. The money is there it is just being funneled in all the wrong ways.

Yeah I just scheduled a technical and they told me it would be leetcode style.

The company I interviewed for directly before was AI only, just prompt and review no coding allowed.

Fun time to prep

I thought this was some weird satire after reading your first two paragraphs.

If you want to convince people of your authority on a topic saying things like: "Nobody's an expert on AI more than me and basically all my friends." is not very compelling.

"Another skill is a communications analysis skill"

It is interesting how people delineate what is a "skill". A 50 word prompt can be called a skill. This process you are describing sounds like it is highly authed and polling or hooked into multiple apps (slack?, text messages?, email?, forums, etc.) and then piping output to an LLM and to generate reports that it pushes to you based on output. You might need some data store to hold all the different communications locally as well.

That is almost a full on app/service that uses an LLM for one layer, but it is still just called a "skill".

I saw a video of a guy in his 50s living in a small city in Vietnam. He claimed $800/mo was more than enough, and $1000/mo would let him live lavishly. He just did the math and as soon as his predicted months of life left * 1000 < savings he pulled the trigger.

I think about that a lot because I have already hit that number easily, my family would definitely suffer though and not be into it.

That is a nice list for someone in their 20s with no responsibilities.

Once you get a partner, kids, elderly parents, pets, mortgage, etc. you won't be worrying about your needs anymore trust me.

I had the exact same description, like verbatim minus coffee and sugar for someone born in 1809.

My paranoid brain thinks the new "Every baby gets a $1000 investment account" is a lame effort on their part to keep it going. For an extremely low price tag they can give every child & parent skin in the game. For someone struggling seeing their kids account go up a $100 might compel them to support the bloated financial/banking industry now that they get a few drops from it.

More like timing.

White guy coming out of college in the late 80s/90s and wanting to get into tech was a rarely opportune hand.

As soon as I read "just redo your career every 2 years" I knew exactly what time period he was 22 during.

Either way a lot of people had similar backgrounds and didn't achieve anything like what he did.

They are only good in the context of the engineer guiding them.

I really can’t imagine what would happen if I didn’t manually intervene sometimes and just kept prompting it for the new behavior I wanted.

Good point, the endless growth means comical amounts of money at the top.

It feels like they discovered in 2008 that you just never let the stock market go down. Then corporations have endless funds and excess can flow freely into politics.

They just have to convince everyone the market going up is good for the average person. The big problem is most people can't afford to invest so they don't give a shit what it does.

Fortunately they are creating a $1000 investment account for all newborns so everyone will be addicted to the market remaining green from birth.

All of these conversational AIs should either confidently detect a users intent or apologize and offer to forward call to a real person.

LLMs perform pretty poorly when you start try to restrict them too much (e.g. detect intents and control responses based on intent detection), the replies gets disjointed and weird feeling. An open ended prompt and chat completion gives a much better feeling most of the time but can go off the rails.

If you don't let LLMs handle the conversation entirely you get into insanely complex traditional engineering problem: (if user is complaining do x, if user asks a relevant question do y, if user asks an irrelevant question do z), which is an endless can of worms.