@kaiwren thanx for shoutout, looking forward to your feedback
10 karma · joined May 14, 2010
Thanx
yes, ollama and Bodhi App both use llama.cpp but our approaches are different. Ollama embeds a binary within its binary, that it copies to a tmp folder and runs this webserver. any request that comes to ollama is then forwarded to this server, and replied sent back to the client.
Bodhi embeds the llama.cpp server, so there is no tmp binary that is copied. when a request comes to Bodhi App, it invokes the code in llama.cpp and sends the response back to client. So there is no request hopping.
Hope that approach do provide us with some benefits.
Also Bodhi uses Rust as programming language. IMHO rust have excellent interface with C/C++ libraries, so the C-code is invoked using the C-FFI bridge. And given Rust's memory safety, fearless concurrency and zero cost abstractions, should definitely provide some performance benefit to Bodhi's approach.
Will get back to you once I have results for these benchmarks. Thanks for the idea.
Hope you try Bodhi, and have some equally valuable feedback on the app.
Cheers.
@levmiseri are you using APIs, or have fine-tuned prompts for each of the location? do share these as well. the github doesn't include this.
you cannot run gumroad like financial transaction business without licence and agreement with stripe. it is bound to be closed down on short notice, and all the earnings will be clawed back.
so don't use this app. period.
good problem to solve, nice personal story, but disappointed it is translating into clickbait+data scraping
wonderful personal story, but doesn't look like it is translating 100% into empathy.
One prefers locking and complicated instructions, the other is designed with open and simplicity.
At one point, Intel seemed to be dominant, and proven right with their proprietary tech, where as ARM chugged along keeping a low profile, and keep doing the good work. Now, finally the reality has caught up with them, and turns out openness and simplicity is finally taking a lead.
Stick to your principles FTW!!!
I remember, I used to keep a diary with landline phone numbers of relatives etc. I used to remember at least a dozen of those numbers for emergency. With mobile and digital contacts, now that number is down to just 2.
Wonder, what the programming is going to look like in 10 yrs. If no-one dedicates so much of their time and effort to document, the programmers then would feel helpless. right?
1. Or at that point, LLMs will be intelligent enough to read the source code and figure out the commands.
2. They can generate data by running commands, and train themselves using this simulated data.
What other scenarios will occur once we stop generating data using human effort because we all are so dependent on LLMs to generate those commands?
also you need a designer. the website looks like from the 2000s.
thanks for sharing the app.
great start, best wishes with the project, looking forward to more amazing demos.
Working in Enterprise, I can share hybrid model works best for multinational. If they are going into new geography, where local laws restrict data transfer across national boundaries, having a cloud provider with local data centers is the easiest and quickest way to launch.
Curious, why not contribute to Citus? Why create a fork and have duplicated effort?
FastAPI is the sweetest and the most intuitive web framework.
I just checked a few doc pages for your framework, seems heavily inspired by FastAPI.
So what have you not found in FastAPI that is making you write a new web framework from scratch?
appreciate if we don't do link drops, rather share it with how it benefits HN audience in mind.
there ain't anything called learn it in x hrs/days