sidmitra

Just this week i was looking for more humour writing like Douglas Adams, PG Wodehouse. I came across this award which seems interesting. https://en.wikipedia.org/wiki/Bollinger_Everyman_Wodehouse_P...

I plan to read some of them this year.

Does your app have a lot of dependencies that do background threads? Like Launchdarkly(feature flags), redis, spyne(rpc) and on and on.

Tests might cover more code than a single unit owned by different teams, thus end up with multiple owners. Prefer "squads" as the owners rather the individuals.

But just like documentation the ownership might be stale and out of sync. So the idea would be let some reds in SLO dashboard correct them over time. It's not possible to automatically link "tests" to the "code" always.

To borrow from medicine: First step is to always stop the `stop the hemorrhage`, then clean the wound, and then protect the wound(or wounds).

- Add a deprecation marker. In python this can be a decorator, context-manager, or even a magic comment string. This i ideally try to do while first introducing the pattern. It makes searching easier next time.

- Create a linter, with an escape hatch. If you can static analyse, type hint your way; great! In python i will create AST, semgrep or custom ones to catch these but provide a magic string similar to `type: noqa` to ignore existing code. Then there's a way to track and solve offending place. You can make a metric out of it.

- Everything in the system as to have a owner(person, squad, team or dept). Endpoints have owners, async tasks have owners, kafka consumers might have owners, test cases might have owners. So if anything fails you can somehow make these visible into their corresponding SLO dashboards.

  The other alternative to this last step is "if possible" some platform squad can take over and do this as zero-cost refactor for the other product squad. Ofcourse the product squads have to help test/approve etc. It's an easier way to get people to adopt a pattern if you do it for them. But the ROI on the pattern has to be there, and the platform squad does get stuck doing cruft thankless work sometimes. If you do this judiciously the win might be thanks enough, like more robust systems, better observability/traces, less flaky tests etc. etc.

^ There's a host of built-in color schemes.

>the fulltext tool, can and should hold only 'active' data

Same can be said about your DB. You can create separate tables, partitions to hold only active data. I assume materialized views are also there(but never used them for FTS). You can even choose to create a separate postgres instance but only use it for FTS data. The reason to do that might be to avoid coupling your business logic to another ORM/DSL and having your team t learn another query language and its gotchas.

> as data size is smaller, it better fits in RAM

> as data size is smaller, it better fits in RAM

> as the indexed data are mostly read-only, the VM where it runs can be relatively easily cloned

> as the indexed data are mostly read-only, the can be easily backup-ed

> as the backups are smaller, restoring a backup can be very fast

Once the pg tables are separate and relevant indexing, i assume PG can also keep most data in memory. There isn't anything stopping you from using a different instance of PG for FTS if needed.

> as FTS tools are usually schema-less, there is no outage during schema changes

True. But in practice for example ES does have schema(mappings, columns, indexes), and will have you re-index your rows/data in some cases rebuild your index entirely to be safe. There are field types and your querying will depend on the field types you choose. i remember even SOLR did, because i had to figure out Geospatial field types to do those queries, but haven't used it in a decade so can't say how things stand now.

https://www.elastic.co/guide/en/elasticsearch/reference/curr...

While the OPs point stands, in a sufficiently complex FTS search project you'll need all of the features and you'll have to deal with the following on search oriented DBs

- Schema migrations or some async jobs to re-index data. Infact it was worse than postgres because atleast in RDBMS migrations are well understood. In ES devs would change field types and expect everything to work without realizing only the new data was getting it. So we had to re-index entire indexes sometimes to get around this for each change in schema.

- At scale you'll have to tap into WAL logs via CDC/Debezium to ensure your data in your search index is up-to-date and no rows were missed. Which means dealing with robust queues/pub-sub.

- A whole another ORM or DSL for elasticsearch. If you don't use these, your queries will soon start to become a mish-mash of string concats or f-strings which is even worse for maintainability.

https://elasticsearch-py.readthedocs.io/en/v8.14.0/ https://elasticsearch-dsl.readthedocs.io/en/latest/

- Unless your search server is directly serving browser traffic, you'll add additional latency traversing hops. In some cases meilisearch, typesense might work here.

I usually recommend engineers(starting out on a new search product feature) to start with FTS on postgres and jump to another search DB as and when needed. FTS support has improved greatly on python frameworks like Django. I've made the other choice of jumping too soon to a separate search DB and come to regret it because it needed me to either build abstractions on top or use DSL sdk, then ensure the data in both is "synced" up and maintain observability/telemetry on this new DB and so on. The time/effort investment was not linear is and the ROI wasn't in the same range for the use-case i was working on.

I actually got more mileage out of search by just dumping small CSV datasets into S3 and downloading them in the browser and doing FTS client side via JS libs. This basically got me zero latency search, albeit for small enough per-user datasets.

That's the second time i'm hearing that quote today from completely unrelated sources.

The quote is from David Ogilvy, in case anyone is wondering.

"Search all the parks in all your cities; you'll find no statues of committees."

https://x.com/The_AdProfessor/status/1705254246109651053

What are character triples? Are they trigrams?

The dispute is seen differently in Chile and is not as simplistic as Chile invading a port. In general i've gotten the sense that the general populace believes that Bolivia(with its secret alliance with Peru) had other intentions.

>In February 1878, Bolivia increased taxes on the Chilean mining company Compañía de Salitres y Ferrocarril de Antofagasta [es] (CSFA), in violation of the Boundary Treaty of 1874 which established the border between both countries and prohibited tax increases for mining. Chile protested the violation of the treaty and requested international arbitration, but the Bolivian government, presided by Hilarión Daza, considered this an internal issue subject to the jurisdiction of the Bolivian courts. Chile insisted that the breach of the treaty would mean that the territorial borders denoted in it were no longer settled.

https://en.wikipedia.org/wiki/War_of_the_Pacific

>Ill-defined borders and oppressive measures allegedly taken against the Chilean migrant population in these territories furnished Chile with a pretext for invasion.

https://www.britannica.com/place/Chile/The-War-of-the-Pacifi...

It helps me keep the config consistent regardless of if something comes from MELPA, or bitbucket/github/gitlab etc.

- I also keep my config in README.org an github which shows the preview by default

- Org mode also helps me keep the commentary alongside. I can even just mark stuff as TODO, COMMENT with keyboard shortcuts. Infact if you read the source instead you'll find parts with COMMENT that don't render in the default preview on browser on Github.

- The git + github bits ofcourse help me use branches/PRs where i can test out stuff. Eg. most recently i was trying out lsp-bridge over eglot.

Here's my literate config: https://github.com/sidmitra/emacs.d

PS: Over time some bits have moved back to init.el to take precedence rather than everything in the org file.

To be able to support that most apps do it as first step is ask for an email to be able to redirect them to the right flow. So the problem is bootstrapping how does a user confirm it's him before he can login to the right system.

Most B2B apps are forced to deal with this because there's no one protocol here, and different paying customers have different internal systems. Asking the user to choose from a dropdown of 20+ paths is proving to be impossible of the extremely high customer support costs.

It's a cycle of misery

The Argentina Paradox: microexplanations and macropuzzles by Taylor et al https://latinaer.springeropen.com/articles/10.1007/s40503-01...

    from box import Box
    movie_box = Box({ "Robin Hood: Men in Tights": { "imdb stars": 6.7, "length": 104 } })
    movie_box.Robin_Hood_Men_in_Tights.imdb_stars


    from munch import Munch
    b = Munch()
    b.hello = 'world'
    b.foo = Munch(lol=True)
    b.foo.lol

https://pypi.org/project/python-box/

https://pypi.org/project/munch/

In my steps above, i was switching the order in between 3 and 4. Adding logic to "thing" first and then updating the test to match; mocking and patching my way across. So implementation first and tests later. Lazy TDD as another commenter mentioned.

- Write the scaffolding test to call the new "thing"

- Add basic assert to check your thing was called with right input

- Add more logic to "thing"

- Incrementally add more asserts to check for output, side-effects etc.

I went and looked at the license. https://github.com/bookwyrm-social/bookwyrm/blob/main/LICENS...

Comment from the author. https://github.com/bookwyrm-social/bookwyrm/issues/2152

My profile is here, as an example: https://bookwyrm.social/user/sidmitra

I've recently moved to Bookwyrm.

I was able to move import my Goodreads list, although many books were missing. Any new books i'm reading, go on Bookwyrm first. I had to add the ISBN details etc. for some. It's a bit of work initially, but atleast i feel my contribution is going to a non-walled garden.

----

https://joinbookwyrm.com/ - Social Reading and Reviewing, Decentralized

>BookWyrm is a social network for tracking your reading, talking about books, writing reviews, and discovering what to read next. Federation allows BookWyrm users to join small, trusted communities that can connect with one another, and with other ActivityPub services like Mastodon and Pleroma.

>Change License: Apache License, Version 2.0, as published by the Apache Foundation.

>Effective on the Change Date, or the fifth anniversary of the first publicly available distribution of a specific version of the Licensed Work under this License, whichever comes first, the Licensor hereby grants you rights under the terms of the Change License, and the rights granted in the paragraph above terminate.

Not the OP, but i didn't know this was a thing.

Although 2 features that would make this perfect are still open

- Private self-hosting: https://github.com/hoppscotch/hoppscotch/issues/870

- sync collections with git: https://github.com/hoppscotch/hoppscotch/issues/870

My use case is that i want to start creating some indexes that are "per-user" and some "per-company" where a company(customer) might have many users. This is to do some sort of double tenant isolation. I will create different keys that have permission to specific indexes and deliver those to the user somehow. My current solution does hacky things with Elasticsearch like adding query filters by user/company-id attributes in the background automatically. But since meilisearch would be customer facing, i need stronger guarantees around permissions per index.

I tried this out a year ago on Meilsearch locally, but haven't stress tested it by creating thousands of them like production.

Or is there a better way to do this. This is also a reason where memory-only systems like Typesense didn't make sense to me. I'm fine with taking a performance hit by going to disk to pull the right index. Not every index will be used all the time. I might also look at sharding/partitioning features if present.

You're right... but just to add.

In any Enterprise project involves a lot of talking to other systems(think Salesforce, Netsuite, Slack) or implementing existing protocols(SAML, OAuth,LDAP) and other such things. Python being popular for very long means there's prior work in almost every area. I believe this can be one definition of "productive", basically re-using something existing and spend your time on other better things. In a lot of the same projects performance at the CPU level might not be a bottleneck because you're stuck waiting 10 seconds for an API call to Salesforce.

I'm more excited about PyO3 and other Python+Rust advances, where it seems you can offload lot of CPU intensive stuff to Rust but keep the Python layer and its ecosystem.