HNHacker News
TopNewBestAskShowJobs

skull8888888

227 karma · joined December 21, 2023

submissionscomments
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
there's def truth in that
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
yep, here's one https://github.com/lmnr-ai/flow?tab=readme-ov-file#llm-agent...
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
here's a good one https://github.com/lmnr-ai/flow?tab=readme-ov-file#llm-agent...
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
Thank you for your comment and wanted to add some clarifications.

1. tasks are not explicitly called from another task In your example greet() is never called, instead task with id=greet will be pushed to the queue

2. The reason I opted for distributed task approach is precisely to eliminate await task_1 await task_2 ...

Going to the point 1, task_2 just says to the engine, ok buddy, now it is time to spawn task_2. With that semantics we isolate tasks and don't deal with the outer tasks which calls another tasks. Also, parallel task execution is extremely simply with that approach.

3. Deadlocks will happen iff you will wait for the data that is never assigned, which is expected. Otherwise, with the design of state and engine itself, they will never happen.

https://github.com/lmnr-ai/flow/blob/main/src/lmnr_flow/stat...

https://github.com/lmnr-ai/flow/blob/main/src/lmnr_flow/flow...

4. For your last point, I would argue the opposite is true, it's actually much harder to maintain and add new changes when you hardcode everything, hence why this project exists in the first place.

5. Regarding deployment. Flow is not a temporal-like (yet), everything is in-memory and but I will def look into how to make it more robust

skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
burr looks very interesting!
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
For me Agent is essentially a decision making machine. I had many iterations on the software around building agents and Flow is the culmination of all of the learnings.

For some reason some LLM specific examples just slipped of my mind because I really wanted to show the barebone nature of this engine and how powerful it is despite its simplicity.

But you also right, it's general enough that you can build any task based system or rebuild complex system with task architecture with Flow.

Signal is clear, add more agent specific examples.

skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
oh, got it :)
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
- AI auditor of Otel traces - wealth manager advisor - AI data engineer to name most interesting cases without giving too much details

but also many chat bots and assistants too

skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
Totally makes, don't know why I missed agent specific examples... will add them asap.

I focused on the Agent because it was initially built for that internally. But you are right, for the release I made it extremely general and to be used as a foundation for the complex systems.

skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
Noted. I had no intention to be disrespectful in any sense. Just pointing out flaws and reasons why and how current project came to be.
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
Thank you for the kind words! Feel free to join our discord https://discord.gg/nNFUUDAKub and discuss stuff there. Also ping me there any time, always happy to help you onboard!
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
Totally agree, I encourage you to give Flow a try! Would be more than happy to help you onboard. Also would love to hear if anything is missing.
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
Also curious, what kind of agents are you building with LangGraph. Would be more than happy to help you onboard to Flow!
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
Thank you! Good question. Behind the scenes, state is manage via Semaphore. So each call to .get(...) acquires a semaphore with a single permit so it essentially guarantees thread-safety. Check out how the state is implemented here https://github.com/lmnr-ai/flow/blob/main/src/lmnr_flow/stat...
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
My main company is Laminar (https://www.lmnr.ai) and we actually help folks ship reliable LLM software to prod. We've seen many of our clients successfully do that. Although, the entire space is very new for sure and things are changing everyday
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
Thanks! Yep, retry and durability guarantee is on the roadmap for sure, I also wanted to start working on the serverless offering. What kind of workflows are you building with temporal?
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
thanks for sharing, looks very interesting! Although, it seems to be very high level and hides all the details behind prompt string. I tend to dislike that kind of things. I designed Flow to be as bare bone as possible and give all control to the user
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
True, graph is not wrong per se, but in my experience I saw a lot of problems, the moment it starts to be a little more dynamic. Main problem is predefined edges. That's where you bump into things like conditional edges, cycles edges and so on. That's also my problem with frameworks like LangGraph. They just get in your way by pushing on things which looked like a good idea in the beginning.
skull8888888··on Show HN: Flow – A dynamic task engine for building AI agents
In some sense yes, task dependencies do form a graph. However, what I meant when I was saying that graph is a wrong abstraction, is that predefined edges is a not the best of writing code when dealing with dynamic systems such as AI Agents.

As you said, the easiest conditional workflows are tedious and we have to rely on "conditional" edges to solve the simplest stuff. Then, how would you define cycles or spawning of the same task multiple times as a node-edge relation? It becomes extremely inconvenient and just gets in your way. The reason I built Flow is exactly because I previously relied on a predefined node-edge system which I also built (https://docs.lmnr.ai/pipeline/introduction).

Also it impossible to do MapReduce like operations on node-edge without inventing some special node for handling that stuff.

Idea about comparing LangGraph and Flow is really good and I will work on that asap! What kind of workflows would you love to see implemented in Flow? Btw, there're a lot of examples in the readme and in the /tests folder

skull8888888··on Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
You are welcome! The easiest way to get started with BAML on Laminar is with our pipeline builder and Structured Output template. Check out the docs here (https://docs.lmnr.ai/pipeline/introduction)
skull8888888··on Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
Thank you! If langtrace sends Otel spans over http or grps we can ingest it! How would one join OTEL GenAI comittee?
skull8888888··on Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
Have you tried BAML (https://github.com/boundaryml/baml)? It's really good at structured output parsing. We integrated it directly into our pipeline builder.
skull8888888··on Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
Thank you! Yes, our pipeline builder is a way to build LLM pipelines and expose them as API endpoints! All hosted and managed by Laminar.
skull8888888··on Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
Hey there, answered similar question here https://news.ycombinator.com/item?id=41453674

We really like langfuse, the team and the product.

Compared to it:

* We send and ingest Otel traces with GenAI semconv

* Provide semantic-event based analytics - you actually can understand what's happening with your LLM app, not just stare at the logs all day.

* Laminar is built be high-performance and reliable from day 0, easily ingesting and processing spikes of 500k+ tokens per seconds

* Much more flexible evals, because you execute everything locally and simply store the results on Laminar

* Go beyond simple prompt management and support Prompt Chain / LLM pipeline management. Extremely useful when you want to host something like Mixture of Agents as a scalable and trackable micro-service.

* It's not released yet, but searchable trace / span data

skull8888888··on Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
Thank you! By that we mean, all processing, i.e. data, forward run, evaluator run is done locally in, let's say, Jupyter Notebook. While it's running locally, it sends all the run data / stats to the server for storage and progress tracking.

We love it because we tried putting things into the UI, but found it to be much more limiting rather that letting users design evals and run them however they want.

skull8888888··on Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
Hey there, by semantic we mean, embedding text and storing it in the vector DB. Regarding scale, we thought a lot about it, and that's why we process span in the background queue. Tradeoff would be that indexing / embedding would not be real-time as scale will grow. We will also use tiny embedding models, which become better and better.
skull8888888··on Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
Hey there, we priced it that way because of our very generous free tier. Your suggestion of usage based pricing also makes sense. So, in our case it might be something: if you pass 50k spans, then you pay something like $0.001 (not a final number) per span. Would you image something like this?
skull8888888··on Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
Yes! Our pipeline builder is pretty versatile. You can define conditional routing, parallel branches, and cycles. Right now we support LLM node and util nodes (json extractor). If you can defined your logic purely from those nodes (and in majority of cases you will be), then great, you can host everything on Laminar! You follow this guide (https://docs.lmnr.ai/tutorials/control-flow-with-LLM) it's bit outdated by gives you a good idea on how to create and run pipelines.
skull8888888··on Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
Now that you mentioned it, > warnings and errors that mislead you

it really makes sense. I guess what I was pointing into, is that when you have really rich text (in your case it would be error descriptions), searching over them semantically is a must have feature.

But you are right, being an output of LLM is not a requirement.

skull8888888··on Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
Hey there, apologies for the late reply.

> Could you clarify why would some need to analyze LLMs out of all the things?

When you want to understand trends of the output of your Agent / RAG on scale, without looking manually at each trace, you need to another LLM to process the output. For instance, you want to understand what is the most common topic discussed with your agent. You can prompt another LLM to extract this info, Laminar will host everything, and turn this data into metrics.

> Why do we need to evaluate LLMs?

You right, devs who want to evaluate output of the LLM apps, truly care about the quality or some other metric. For this kind of cases evals are invaluable. Good example would be, AI drive-through agents or AI voice agents for mortgages (use cases we've seen on Laminar)

← PreviousPage 2 of 3Next →