mhamann

Reliable timing across TX/RX was by far the biggest hurdle. Raw USB access and overall throughput really hasn't been as big of an issue as I thought it would be.

Can you help me understand MetalRT a bit more? Based on the name, it sounds like something that's Apple-only (although, Apple basically co-opted the name Metal, which was traditionally more generic). Does or will MetalRT run on more platforms?

What about MetalRT's relationship to llama.cpp, onnx, MLX, transformers, etc? Is MetalRT a replacement for those? Designed to be compatible with a wide variety of model formats? Or are you just providing an abstraction on top of these?

Super interesting. Auth and Payments seem like things you don't want to or can't build yourself. (I'm waiting for the day when I can just use BTC or UDSC for everything.)

Have you thought about supporting additional auth providers? Or providing a way for other auth services to add support for their products?

I do really appreciate you taking the time to drop by and leave a comment. But I'm curious...why do you think building agents is so important vs. building more of the "AI infrastructure" (which is really what LlamaFarm is trying to do) that will enable the devs that are building integrated AI systems (including agents).

Appreciate the pep talk, but let’s not pretend infra and developer experience plays can’t scale. GitLab, HashiCorp, and Vercel all "just built better DX for open source" and somehow ended up billion-dollar companies.

Agents will come and go (and probably run into the same orchestration headaches), but someone still has to build the reliable, open foundation they’ll stand on.

Multiple areas of degradation. Typically, you don't ship a dataset to prod and then never change it. You want the system to continue to learn and improve as new data is available. This can create performance issues as the dataset grows in size. But also, your model's performance in terms of quality can degrade over time if you're not constantly evaluating its responses. This can occur because of new info within RAG, a model swap/upgrade, or changes to prompts. Keeping all of those knives in the air is tricky. We're hoping we can solve a bunch of pain points around this so that reliable AI systems are accessible to anyone.

Yes, our goal is to provide a stable, open source platform on top of the cutting-edge AI tools. We can systematically update dependencies as needed and ensure that outputs meet quality requirements.

We also have plans for eval features in the product so that users can measure the quality of changes over time, whether to their own project configs or actual LlamaFarm updates.

Yes, all that's a bit hand-wavy, I know. :-) But we do recognize the problem and have real ideas on solutions. But execution is everything. ;-)

`lf deploy` here we come!

Oh! Muna looks cool as well! I've just barely glanced at your docs page so far, but I'm definitely going to explore further. One of the biggest issues in the back of our minds is getting models running on a variety of hardware and platforms. Right now, we're just using Ollama with support for Lemonade coming soon. But both of these will likely require some manual setup before deploying LlamaFarm.

Great point. I can see how you'd land there. Also a great idea! xD

Maybe a better descriptor is "self-sovereign AI?" "Self-hosted AI?"

I think that enterprises and small businesses alike need stuff like this, regardless of whether they're software companies or some other vertical like healthcare or legal. I worked at IBM for over a decade and it was always preferable to start with an open source framework if it fit your problem space, especially for internal stuff. We shipped products with components built on Elastic, Drupal, Express, etc.

You could make the same argument for Kubernetes. If you have the cash and the team, why not build it yourself? Most don't have the expertise or the time to find/train the people who do.

People want AI that works out of the box on day one. Not day 100.

Right...there are lots of ways you could do that. Most of the ways we've seen enabling that sort of thing tend to be programmatic in nature. That's great for some people, but you have to deal with shifting dependencies, sorting out bugs, making sure everything connects properly, etc. Some people will want that for sure, because you do get control over every little piece.

LlamaFarm provides an abstraction over most (eventually all) of those pieces. Something that should work out of the box wherever you deploy it but with various knobs to customize as needed (we're working on an agent to help you with this as well).

In your example (alarm monitoring), I think right now you'd still need to write the agent, but you could use LlamaFarm to deploy an LLM that relied on increasingly accurate examples in RAG and very easily adjust your system prompt.

Cool idea. Thanks for sharing. I was really annoyed by the way Google nerfed the maps timeline stuff last year. Obviously this project is way more ambitious than that, but just goes to show you how little Google cares about the longevity of your data.

Thanks! It means a lot to hear you say that.

100%. We don't know what's going to happen in the future. Things are evolving so quickly. Hopefully pushing back on centralization now will keep the ecosystem healthier and give developers real options outside the big two/three cloud providers.

I love what they're doing in expanding the program and such, but the one problem that all "startup tiers" seem to have is that they only last one year.

Most startups are not profitable after just one year and paying cloud bills is a drag on growth.

A graduated drop-off would be better maybe 100% off for year one, then 75% year two, 50% year three, 25% year four, and full price year five. All subject to some sort of annual or total cap, of course.

That's a more realistic picture of what it takes to get off the ground.

Auth0/Okta uses jwt.io to promote themselves, so why shouldn't we do the same? ;-)

But this was also more about scratching an itch that we had internally where similar tools didn't support all of the features we wanted them to. So, we wrote our own.

Thanks for checking it out!

It's definitely more than an afternoon of work, though I can understand how you might think that. Things are often harder and more complex than we realize at first. :-)

One of the biggest gaps with the current implementation is lack of support for JWKs. We wanted to support that. Also, dark mode. It's the little things.

For the record, no tokens are ever sent anywhere. Everything stays local to the browser. If you hit the "GPT" button, it just transmits the payload--nothing else. That part is definitely a fun experiment. It's more "useful" with more complex payloads. But it may not stick around long-term. We'll see.

Thanks! We have a few improvements to make around copy/paste stuff, array printing, etc. Thanks for the feedback!

Sorry about that--always a few bugs in the first rev. It's working now!

I wonder this as well. I've never come across a real-world implementation of PASTEO.

https://jwt.is is also fully client-side

Thanks!

Yeah, the GPT stuff was just a fun easter egg. :-D

Yes, it certainly could be; however, it would be up to the maintainers of that project whether to accept contributions. "Competition" is good for everyone though. Maybe they'll adopt some of our ideas. :-D

I have to admit that I'm not up to date on what's out there in this space, but this seems like a pretty useful tool.

Is there or will there be a UI to show all of the things you're monitoring?

As the only SQL client with native support for PlanetScale, Arctype is definitely useful from that perspective. I also like the UI and find it pretty easy to use.

In my admittedly limited experience using it, I found that some MySql datatypes seemed to be missing from the table editor (e.g., `TEXT`). Not sure why this is and I haven't had time to dig further.

It's promising, and way better than even some of the other paid options. (Like, what is DBeaver even doing??)

Congrats on the launch, Adam!

We're using Kable at Rownd to better understand how our customers use our authentication and account management services. For example, we can track how many sign-ins a particular customer is driving through their website and whether their users typically use email or SMS to sign up. Eventually, we'll be able to use this data to bill our usage-based customers for their volume.

Overall, implementation was about as simple as stated above. In our Node.js backend, we imported the Kable library, created an instance with our tenant config, and then added one line to the API methods we wanted to record.

Thanks for the feedback. It's not you, it's us. ;-)

We've just been making some updates to our home page to hopefully make it clearer, but we realize we still have plenty of work to do.

If you sign in at the Rownd website, you can experience how the product will work on your own website.

We'll definitely get a shorter demo video posted soon that helps explain this too.

Thanks for the feedback! We can definitely produce a shorter demo video that provides a basic walkthrough of how it works without going into tons of detail. The one we produced for this was more of a live-coding dev walkthrough.

As someone who prefers reading text over watching videos, I can commiserate with you ;-)

Thanks for bringing up this point! I think there are definitely times where self-hosting authentication might make sense, but I'll respectfully disagree that most people should do this.

Every component that gets added to your infrastructure is just "another thing" that you have to worry about in terms of uptime, monitoring, security, staying current, and so on. Personally, I'd rather not worry about any of that for something that isn't part of my core competency. Certainly there is a cost/benefit analysis to be made, mostly for larger companies.

I know we won't likely agree on this point, and that's ok! I just wanted to share an alternative perspective. :-)