probst

Thank you for the link to your harness! Looks quite interesting.

What type of ongoing tuning do you do for your model+provider+harness combo, and when do you know it's time to redo it? Or more specifically, when you do the tuning, how do you actually go about doing it?

I am mostly using omp (omp.sh) which is very tunable, and I did some tuning for one of the GLM models in the past (mostly based on vibes because people said it was getting dumb like a brick after some context lengths so I had omp compress sooner).

Yeah, good point. For HTML/CSS/JavaScript it seems to do fine. Maybe I am just asking too much of it, with a codebase spanning Rust and Elixir and doing some more low-level stuff in-between...

Last year I didn't use any of the model (other than for chat and some snippets here and there), exactly because I didn't think them good enough. It should be said that I think DS is vastly better than that, but I think the closed source frontier models are just so good by now, that stepping down to "N-X" months just feels all the more painful.

Ah, interesting use cases for DS! I haven't had these needs yet, which maybe also explains why I haven't considered using DS for them. Thanks!

Would you please elaborate on "really decent"? Really decent for what use exactly? I want to love it (and use it as a cheaper research model in my harness), but I really cannot get any real development done with it? It stumbles and fumbles and over-complicates, where astra just plows ahead (often in less overall time) with excellent solutions.

I really want to love this model, particularly since it's good to have an open weight model on my team so to speak, but I think I might be holding it wrong.

What are you using it for?

Exceptionally well is my takeaway. It’s the only harness I am using these days. I was previously using pi and codex mostly, but also the ones built into editors like zed, vscode, and the jetbrains IDEs.

On top of that, somewhat unrelated I’ll agree but still, it has support for vim keybindings

Way to restrictive in terms of tokens provided. I am on their largest plan, and quickly run into their limits. And that is using it selectively in addition to codex.

I have been a long time subscriber to the economist for the same reason. The print edition is great and keeps me feeling informed. If only it wouldn’t include the digital edition too… Invariably, the more stressed I get in my everyday life by work and small children and lack of sleep, the more I get dragged towards the online edition of the economist, and BAM I’m back in the constant instant news cycle again, even though the site doesn’t update that often.

My latest life hack which I can recommend to anyone, at least in Germany: as part of my local library membership I get access to the PressReader app, which gives me access to the print edition of the economist! At 10 EUR a year it’s so affordable it nearly feels like it’s unethically so for what I get. I guess I’ll close with: May we forever be blessed with libraries, and may they never be „optimised away“ as an unnecessary social good.

Yes, I was so close to signing up for a pro account to get some inmail credits to be able to write someone directly, but then I decided to try writing here first lest I be stuck in a pit of dark patterns trying to cancel my subscription. I would however have been my next step.

I finally heard back! Thank you! I think this thread might have helped!

And I totally agree with your thoughts on the criticality and inadequacy of AI replies in such a context! Lack of answers was holding a months worth of income hostage with no real recourse for us other than trying to be loud online. Kind of sad.

Looking at alternative merchant of records now to buy us some optionality.

Taking a peek at the repo and the local first data layer issue (https://github.com/bartblast/hologram/issues/940) I get the sense that you aren't (and not planning to) using any off-the-shelf CRDT or the like, am I right in that assumption?

How will conflict resolution be handled? Or is it a last-write-wins type setup? It would be amazing to have this build on top of automerge or something like that! It could compile to and use the automerge js libs in the browser, and deal with automerge through a NIF server side? Ok, not knowing your implementation and plans, I have to confess I am talking a bit out of my arse here...

Exactly, or rather, the social habit part I agree a 100% with. I don't want to train myself out of being a polite human being, just to save some keystrokes.

I certainly say please and thank you. Whether or not it makes a difference to the LLM, it makes a difference to me. I want to retain some humanity and politeness in my own behavior, even if I spend an inordinate amount of time communicating with, instructing, and debating a non-sentient piece of code.

Heh, nice app! I'll check it out!

I'm all for word of mouth and things radiating out, but doesn't that need some viral factor of sorts that makes the product one that's shareable? We have customers raving about our product actually, and saying they would be incredibly sad should it go away, but are struggling to leverage that into further sales...

Affiliate marketing _does_ work to an extent, and is a kind of rideable wave I suppose?

Do you have concrete examples?

Awesome, thank you!

Thank you for your thorough reply! I think we should give it another go and be more restrictive in targeting!

Speaking of paid ads: we tried google, bing, LinkedIn in the past. Do you have a preferred place that you recommend that you have seen better performance at than other places?

Ah, the research angle is good! And my take-away from what you write is mostly the need to be visible, in whatever form, be it in the users communities, or otherwise?

We have excellent feedback from the users who do find us, so there is product market fit for the subset of customers that have a need for our product strong enough to actively search for it, but maybe that's an issue in and of itself: the fraction might be too small, and furthermore, for the (hopefully) larger segment of the user base that don't have a need strong enough to actively go looking, we need to be sufficiently visible for them to chance upon us...? which I guess drives home the message even more that visibility is key...

Thanks!

Long story short: Meilisearch is a very good search application for sure, but our needs drifted apart.

Longer version: Our usecase isn't entirely aligned with how Meilisearch is meant to be used, I think. For one, we have a very high number of indices (at the time of using Meilisearch we had multiple per site that uses our service) and we wanted to have these replicated to edge nodes we have placed around the globe to keep latency down. In the end I was ending up spending an inordinate amount of time wrangling Meilisearch to get indices up and running across our different edge nodes. And, I am not sure if it was a regression in the last Meilisearch version we used, or user error, but at some point some index creations would just never get done, and would hold up our entire indexing pipeline. So long story short: the amount of effort needed to orchestrate and keep all the individual Meilisearch nodes up to date and in sync just turned out to be too much hassle.

The system I built afterwards works vastly better because it's actually bespoke and tailored to our use case. I can create the search indices on our crawler nodes entirely in isolation, run all sorts of preprocessing and optimizations once, and then just ship the final search indices to all the nodes that then serve them, and as a result I need wastly less computational resources on those edge nodes too, since no actual indexing work ever needs to be done on them.

It also means (double edges sword of course, because Meilisearch already gives you so much functionality out of the box) that I had to re-implement a whole lot of stuff around relevance tuning, snippet generation, ranking, etc, but as a result I have also been able to build interesting functionality than we didn't get out of the box before. And lastly, since the search index is now served as part of our main search node application things also get really fast, as I have no latency whatsoever to speak of going from app to Meilisearch.

We are working on our two site-search search engines Monocle Search [0] and SearchCue [1]. The former is squarely aimed at people using Squarespace, whereas our newer offering SearchCue is aimed at anyone with a site that wants to add search.

We used Meilisearch as the search backend in the beginning but have since replaced it with a quite sophisticated search stack built around Tantivy [2]. We now support crawling and indexing of pages, most common office documents and PDFs, run OCR and feature extraction of images you might have, offer typeahead search with the aim of giving you providing answers as fast as you can type, as well as more classic agentic/conversational ai search.

There have been quite a number of interesting optimization challenges to solve in case anyone is interested. We have search nodes distributed around the globe to provide the lowest possible latency regardless of where the end-user sits.

We are also working on some other smaller side projects, but they aren't quite ready to launch yet.

[0]: https://monocle-search.com

[1]: https://searchcue.com

[2]: https://github.com/quickwit-oss/tantivy

Or tell pi to add support for the coding plan directly. That gave me GLM-5.1 support in no time along with support for showing the remaining quota, etc, too.

It also compresses the context at around 100k tokens.

In case anyone is interested: https://github.com/sebastian/pi-extensions/tree/main/.pi/ext...

I am really curious about LiteSync. Any chance you could share a bit on your experiences with it (recognising it’s somewhat off-topic…). Do you run with multiple primaries? What sort of use cases do you reach to it for? Conflict resolution seems a bit simplistic at first glance (from the perspective of someone very into CRDTs), have you experienced any issues as a result of that?

I am also based out of Germany, and experience the lag too. I find it very annoying actually, to the point where I am contemplating switching. The lag isn't always there though...

Yeah, that is a great idea! I tried building something like that, but didn't quite get it right. Maybe I should give it a second go.

I have also thought about adding a feature whereby you can ask the equivalent of "I would like to be able to pay myself X per month" and then be told how much you would need to bill per month. In a way it's the inverse of what the tool does today.

Thank you :)

The problem has now been fixed. Fields can be emptied properly, and the TER (decimal values) are editable too!

Thanks again!

You are absolutely right! Thanks for letting me know! I’ll look into it when I get home!

Thank you for sharing! That is even more strict than what my tool calculates, as in 75% set aside for the future. The exact numbers likely also depend on where you reside, and how much of a social security net exists to catch you when things don't go as planned.

An incredible level of polish and ease of use.

I am not sure what your background is, or whether you are, or have worked as, a freelancer. What I am writing might therefore be old news to you, in which case it would still be interesting to hear how you are thinking about the numbers I'll present.

A lot of people write comments such as yours. 11000USD/mth is a very decent salary if you are an employee. What most regular employees completely discount when doing back of the envelope calculations about the salary of other people are the amount of additional contributions that are made by your employer that regular employees might never see, but that are paid for you all the same.

Here is the math for me as a freelancer in Germany, with a similar salary as the one you are claiming is outrageous.

I bill roughly the equivalent per month from Berlin in Germany. I am not sure where the Babel developers are based, so their circumstances might differ, but all the same, here goes:

Of the 9000EUR (11000USD in EUR) I have to pay: - 19% VAT = 1750 - 950/mth in health insurance - Put aside 1000/mth for retirement (which is less than I previously had from my employer) - Set aside money for vacation (as a regular employee in Germany I would have 30 days paid vacation, as a freelancer I had to set aside roughly 600/mth to compensate for this) - Set aside money in case I get sick (as a regular employee I still get my salary when I am sick, as a freelancer I get paid from day 45). I set aside roughly 450/mth for this. - Then there are miscellaneous freelancer related expenses and having to buy hardware (which your employer would cover too). Let's make that 300/mth. - Then there's income taxes on the rest. This differs from jurisdiction to jurisdiction, but let's make it an even 30% (which is below what I pay).

This leaves me with 3950 before taxes, or 2765 after taxes... That doesn't sound so amazing anymore, does it?

Even if the Babel guys aren't working as freelancer but pay themselves through a company, the math works out roughly the same (at least from what I have seen in my own business over the last 9 years). The only difference is that the company needs the equivalent monthly budget instead of the individuals.