everlier

So many comments about reasoning here, yet none about the very obvious one, it's not stability of the infrastructure, it's future direction of a product like Claude Code. They need to know how to continue their optimisation machine to fit developers needs the best way possible (for good or for worse).

I guess we should wait for some opt-out telemetry some time soon. It'll be nothing too crazy at first, but we'll see how hungry they are for the data.

I was, for a long time, scared of my future due to the low/no-code, automation, LLMs, outsourcing, etc. Until, at some point, I realised something simple - the risk factor for my job is not determined by how good new tools are, but only by how lazy people are about learning and adopting them. And here history gives another lesson - we never learn, eternal cycle of mistakes will continue.

I feel like there's a story behind multimodal architectures not being integrated into the main package by default

Nice use of Flutter!

I'm getting disillusioned with the big clouds, all of the reports about account hacking, 20m to find how much you have to pay for a thing in a thing you didn't know that needs to be paid for...

Co-location or a VPS seems like a best choice in all cases except when you're an enterprise.

To save a click, in 2022, the distribution was: Linux - 46.25%, MacOS - 32.21%, Windows - 21.54%

There was another technique "klmbr" a year or so ago: https://github.com/av/klmbr At a highest setting, It was unparseable by the LLMs at the time. Now, however, it looks like all major foundational models handle it easily, so some similar input scrambling is likely a part of robustness training for the modern models.

Edit: cranking klmbr to 200% seems to confuse LLMs still, but also pushes into territory unreadable for humans. "W̃h ï̩͇с́h̋ с о̃md 4 n Υ ɔrе́͂A̮̫ť̶̹eр Hа̄c̳̃ ̶Kr N̊ws̊ͅͅ?"

Their agents can natively read files and webpages from URLs. It's so convenient, I've implemented identical feature for our product at work.

Mostly depends on which features are most important for you. I'm a SWE, so using their Assistant for Web RAG nearly exclusively for work-related stuff and for most of my personal queries. I'm rarely using multi-modal content, mostly sticking to text. They support many providers and new notable models are typically rolled out only a few days after release which is always great to test out. I have a standalone subscription for coding agent LLM. If above aligns - might be a good choice.

How do you like bugs where tools are not working, but only for Ollama provider and only when streaming is enabled? This is one of the real instances I had to debug with LiteLLM.

Training or integration?

I work on Harbor (https://github.com/av/harbor), it is a project to save hours when setting up LLMs locally.

I pre-integrated over 50 different LLM-related projects, added a nice CLI and a Desktop app on top to manage the configs.

If my understanding is correct, this only makes sense for codebases that do not fit in memory of a largest build box an organisation can run

Opus 4.1 is not great at writing articles and LLMs are terrible at generating new ideas.

I can't wrap my head around possible yields, as the method relies on diamond crystals forming in the heat-conducting pillars within the chip, so if the process less than perfect - it can be a source of delayed failure from termal issues within the chip. It also look like a heat-conducting grid would further decrease usable space and the whole wafer needs to be designed around it.

That said, mentioned temperature gains are absolutely and utterly insane even if they come with some high-frequency issues.

Something like https://github.com/av/klmbr could also work

This is a "Deep Dive" podcast generated with the NotebookLM using information from Harbor's Wiki.

Harbor is a containerized LLM toolkit that allows you to run LLMs and additional services via a concise CLI.

Repository: https://github.com/av/harbor

Harbor is a toolkit for working with LLMs locally, it got an upgrade recently - an LLM proxy (called boost).

It comes with first-class support for streaming, multiple downstream APIs and custom modules, aka scripts you can write to bash some workflows together.

One of the most interesting built-in modules is klmbr, which can inject entropy into your prompts, making LLM outputs much more diverse as a result.

I had the same situation, maybe gods are angry at us due to some reason.

Building a toolkit to save hours of time when working with LLMs locally. For example, adding a TTS or Web RAG to the Open WebUI setup both are a single command.

https://github.com/av/harbor

txtai accomplished a similar task by fine tuning a very small t5 model, notebook with usage samples (training code has to be somewhere near)

https://github.com/neuml/txtai/blob/master/examples/33_Query...

EVRYTHNG Platform docs are extremely thorough and versatile. There're also lots of guides and examples covering how to work with their platform. Taking into account complexity and amount of abstractions, those are one of the best docs i've ever read.