Agency: Pure Go LangChain Alternative | Hacker News Reader

Agency: Pure Go LangChain Alternative

github.com

77 pointsemil1439 comments

I like Go a lot for working with OpenAI etc, it's 'just' API calls and Go is great at that. I've opensourced some bits here: https://github.com/stillmatic/gollum -- in particular, function dispatch (given a prompt, return an arbitrary Go struct) is really nice, as is a very fast in-memory KNN index.

This is cool -- I wrote something a bit similar here but uses just raw Go as the source input: https://github.com/tmc/go2oapi

Correct me if I'm wrong but looks like this is a helper-utility to work with openai API. Agency on the other hand gives you tools to abstract away such things and write high level code

definitely serve different purposes, I basically think abstracting away the flows (agency/langchain) is the wrong layer of abstraction, and that it's more useful to abstract away certain problems to solve (eg vector retrieval) or useful patterns (eg function dispatch). another comment mentioned that langchain had hardcoded prompts, which make it difficult to experiment and iterate (easier today, perhaps, when I first used it, it was certainly a mess). where is the complexity hidden?

imo using someone else's abstractions for your entire workflow is a nice way of making a hackathon project, but a terrible way to build a full product.

> Using someone else's abstractions for your entire workflow is a nice way of making a hackathon project

We use someone else's abstractions all the time, when they are good :) Think of reactjs, expressjs, django, etc.

> Abstracting away the flows (agency/langchain) is the wrong layer of abstraction

But you do abstract away SQL when you write business logic isn't it? Many people uses ORMs for that, which gives ability to easily switch databases and write tests. Why do you think this isn't the same?

> langchain had hardcoded prompts, which make it difficult to experiment and iterate

Agency does not and won't have anything like that except it's going to make a LOT of sense. Even then there 100% will be to control that stuff.

> Where is the complexity hidden?

The biggest complexity was to design the library to be small, simple and extensible. It's not a lot of abstractions out there.

But the better answer to your question is probably something like - "complexity is hidden behind how providers implement core interfaces"

yeah, I just think it's a question of picking the right set of things to abstract away (and we're all figuring out what set that should be). I chose to not add much abstraction over the go-openai library to support maximum customizability, and better support as openai adds functionality. I think the custom abstractions (eg Message, Operation, etc) are helpful iff you add additional non-openai providers.

fwiw I really like agency's interceptor funcs, good place to add tracing, metrics, logging, etc.

> you add additional non-openai providers.

100% will do, just not for the initial release

> fwiw I really like agency's interceptor funcs, good place to add tracing, metrics, logging, etc.

thank u much, glad to hear it :)

See https://news.ycombinator.com/item?id=38475745 / https://github.com/tmc/langchaingo for a closer-to-langchain alternative.

Unfortunately, you're using sashabaranov/go-openai, which has different api behavior than the official langchain python package [0].

I was hoping this would be a drop in replacement for langchain (so DS ppl can code in python and gophers can port it to production).

[0] - https://github.com/sashabaranov/go-openai#why-dont-we-get-th...

1) Thank you for pointing the moment with 0 temp, I think we can make `agency.TextToTextParams{Temperature *int}` where nil gonna be passed down like `0` and 0 gonna be mapped to `math.SmallestNonzeroFloat32`. We also will add the ability to use seeds though

2) "I was hoping this would be a drop in replacement for langchain" - why don't use use langchaingo then? The thing is - we don't like langchain much. To is it looks too complex, we thing we can do better, simpler. The final goal is to make a library that can replace langchain and/or autogen but it's definitely not gonna be "drop-in" replacement

You might be interested in https://github.com/tmc/langchaingo which adheres to some of the conceptual modeling in langchain.

UPD: bug with temperature fixed in master

Small core, simple api, openai bindings

Any upsides to using this vs writing 30 lines of code?

There's no license on this project so the default is All-Rights-Reserved so no one aside from the original author can use it until then

github's Original Sin. we could have so much more software training data if they just defaulted (defaulted! you can opt out instead of opt in) to an oss license. so much software is meant to be open source but pple just forgot/didnt bother/didnt know to slap a license on it

I disagree with opting people into a license, because the choice of _which_ license is a very personal and important choice. Defaulting to a very permissive license would make folks think that more restrictive (but user-benefitting) licenses like the GPL are Bad ™ when they're not, whereas defaulting to a GPL flavour would make a load of things more restrictive than they may want to be. The only person who can make that decision is the author.

However, I think one thing GitHub / other source forges can do is make it _clearer_ when no license is present and that it defaults to All-Rights-Reserved

Thank you, fixed, MIT license added

I'm using Semantic Kernel (https://github.com/microsoft/semantic-kernel) and it's really nice. Makes building more complex workflows really simple without sacrificing control.

A bunch of examples (https://github.com/microsoft/semantic-kernel/blob/main/dotne...) for how to handle just about anything you need to do with OAI with a lot less boilerplate.

yeah but it's not go (and go-centric)

1) your 30 lines could become 10 lines or something

2) you can write abstract code without explicit dependency on openai and easely switch to e.g. anthropic in future without rewriting the whole thing (think of ORM for LLMs)

3) ready to use abstraction like Process that allows to chain operations (more complex stuff is much easier than writing by hand)

I asked the same thing for langchain

Langchain is really bad. You’re so much better off just doing whatever you need to do yourself, especially when it comes to interfacing with OpenAI APIs. It has all of the baggage of a big system without any of the benefits.

Indeed, after being frustrated with existing frameworks, 7 months ago, we started buidling Langroid[1] Multi-Agent LLM framework. It has an elegant Inter-agent orchestration mechanism[2], among many other things. We’ve taken a measured approach to avoid bloat and excess abstractions. We have a company using it in production to assist contact-center/customer-support agents.

[1] https://github.com/langroid/langroid

From the README:

Langroid is an intuitive, lightweight, extensible and principled Python framework to easily build LLM-powered applications, from ex-CMU and UW-Madison researchers. You set up Agents, equip them with optional components (LLM, vector-store and tools/functions), assign them tasks, and have them collaboratively solve a problem by exchanging messages.

[2] Docs on Task Delegation https://langroid.github.io/langroid/quick-start/multi-agent-...

Lol, yeah the hidden prompts in langchain, and the bloat turned me off it pretty quickly.

Still kinda nice to get something going in less than a day

We think that too! Agency won't have implicit prompts. Or at least without explicit and easy way to override them

I don't mind that they're hidden. I mind that they can't be overriden.

I mind. Langchain is full of shitty, leaky abstractions and shallow function calls. It's a pile of bullshit that somehow managed to raise millions of dollars due to hype. It's everything that's wrong with the state of development in the AI space.

that smells wrong. are you quite sure they cant be overridden? or subclassed?

It looks like the two instances I was thinking of have been fixed:

https://github.com/langchain-ai/langchain/blob/master/libs/l...

I think the intention behind langchain is good, but the implementation suffers from bad design. We tried to fix that by having small core with as less abstractions as possible.

Think of ORM for LLMs. That's not the whole purpose of "langchain-like" frameworks but on that is easy to grasp

Looks nice, although method chaining is not really idiomatic Go and is generally discouraged.

Method chaining is a coding idiom. It’s kind of silly to say an idiom isn’t idiomatic. I know I’m being pedantic though.

Still if you tell say something is generally discouraged you should say why. In this case it’s discouraged because it doesn’t mesh well with go’s error handling.

Actually error handling is much easier with agency's Processes (pipelines)

Instead of writing:

result, err := operationFoo(userInput) if err != nil { panic(err }

result, err = operationBar(result) if err != nil { panic(err }

You can create a Process and handle errors only once:

result, err := agency.NewProcess(operationFoo, operationBar).Execute(userInput) // ...

Imagine having 3+ operations, like 5 or 10. That's gonna be some real difference.

All the chained methods like `SetPrompt(...)` returns `agency.Operation` and no error needs to be handled there

I would, at the very least, wrap the errors being returned inside the process function https://github.com/neurocult/agency/blob/14b14e50a7570189388...

Or, I suppose the user must handle exception behavior in their custom `OperationHandler`

1) You said about process but pointed to operation

2) What wrapping would you add out there? (Wrapping must be added in provider's implementations though)

3) No, user shouldn't and can't handle errors by implementing operation handler. To handle error it simply uses go's native mechanism "v, err := ... ; if err != nil", it just needs to be done once instead of doing so for every operation in the process

Thank you for your comment :)

Why do you think that "method chaining is not really idiomatic" or how this should be implemented?

Reply on news.ycombinator.com