Hacker News Reader

Comment by wkat4242 | Hacker News Reader

Hmm most of the things I used search for are now covered by ChatGPT (for which I use the api which costs cents).

I don't think web search is still important enough for me to pay for. A year ago it would have been. I originally signed up with kagi but I wasn't too impressed and then they came with the limited plans and that was it. Not sure if I'll go back now.

And yeah like the article says it was priced for "silicon valley bros". Even $10 is a lot on a Spanish salary still. But it's doable and 300 for $5 is a decent deal IMO. I wouldn't do that many.

View on HN

what do you use to hit the api? is there a good CLI tool for linux?

Not the op but I use chatblade[0] on the cli, chatgpt-next-web[1] as webgui and quivr[2] for multimodal stuff files/images/audio/video. atm everything goes over a azure openai endpoint but would love to infere an llm locally.

[0] https://github.com/npiv/chatblade

[1] https://github.com/Yidadaa/ChatGPT-Next-Web/

[2] https://github.com/StanGirard/quivr

Is there much difference between the Azure endpoint and the direct OpenAI endpoint?

I really wonder this too.

At work we're always told to use the Azure one because our CIO is elbow deep in Microsoft's ass.

But I wonder.. one funny thing is that everyone at Microsoft is referring to OpenAI as an 'acquisition' whereas as far as I know they only have a <50% interest?

It's mostly speed and that you can disable the constraints openai uses on their API and chatgpt. You have to write a motivational letter though...

I use the ChatGPT bot for matrix mainly. I've tried some other frontends that are more like ChatGPT web but I keep going back to matrix because it's just so handy. I have all my other chats in there too, through bridges.

I use the OpenAI playground because I'm paranoid that third party frontends will steal my API keys and I don't have enough time to audit the code or set up firewall rules.

You can set up a spending limit on the API anyway. It's pretty low risk IMO.

chatgpt-cli is a great tui

Simon Willison’s `llm` is an excellent command line client, and now has `llm chat` as well as `-c` for ongoing conversations.

https://llm.datasette.io/en/stable/

If it's useful, I recently open-sourced a simple GPT-4 TUI [1], which I use many times a day. And since this is a Kagi post, I might as well plug my (extremely simple) Kagi CLI [2], which uses their FastGPT API under the hood.

[1] https://git.sr.ht/~bsprague/gpt4-tui

[2] https://github.com/bcspragu/kagi

so is that better than using the chatgpt subscription?

Much better imo and much cheaper unless you do 200 chats per day or something :P

Why is it better? More integration capabilities. You're not limited to the webpage and app, you can use it anywhere you want. Not constantly kicking me out to log in again every few days (their web does this). The ability to select the model and temperature (a measure of how creative the LLM is). Also the default model seems to be much more recent.

And the price. If you use it sparingly you might be paying as little as 20 cents a month instead of 20 dollars. Not exaggerating but it depends on the size of questions and responses. But really to make it cost as much as that 20$ per month with the API you really have to go hell for leather with it.

I also really like that I’m learning tooling that lets me easily switch model provider later.

If you're even a moderate user of chatGPT, the equivalent cost of using the smallest GPT4 model would run you a hell of a lot more. Let's actually run the numbers.

The cost of the GPT-4 API is ballpark around $0.05 / 1000 tokens. If you want to include a rolling context window, you will easily hit 1000+ tokens if not a huge amount more.

ChatGPT pro gives you 50 GPT-4 queries every three hours. If you're using it all day you might average about 100 daily queries. Using a dedicated GPT4 API would run you approximately five dollars a day for the same thing - that's $150 a month as opposed to flat cost of $20.

How hard is it to hit 100 queries in a day? Pretty damn easy when you realize that most queries aren't usually standalone - instead they involve a back and forth approach which necessitates a rolling context window and you explore the problem space.

When a query (plus back and forth) might cost as much as $0.25, paying for GPT-4 via the API would only net you a whopping 80-100 queries per month before you exceeded the cost of a pro subscription.

The API certainly has its advantages over the web based ChatGPT, but price is definitely not one of them.

Fair enough, but I generally use GPT-3 (3.5). Because I didn't really see much benefit from GPT-4 when I had the subscription. I really didn't find it a whole lot better. The one thing I did use was the bing web lookup but they disabled that, that's when I cancelled my subscription.

I really hate the web interface also. It's constantly kicking me out, so I have to log back in and rewrite my query. And it keeps switching back to GPT-3 which is one of the reasons I ended up using that. Using it with my own tooling is just so much better.

And I don't really use it that much. I just don't have a lot that I need it for yet. I definitely don't even get to the 100 a month :)

Also I don't use it as hard every day. On the weekend I don't tend to even touch it. But you're right, your mileage may vary wildly here.

Good point though I should revisit GPT-4 again. See if it's improved.

Thanks for this insight! Based on this, I am better off with the ChatGPT monthly subscription. I tend to use it a lot with a lot of back and forth. I regularly hit the 50 GPT-4 queries.

I don't even bother using GPT-3.. it's not good enough for my use.

Reply on news.ycombinator.com