Hacker News Reader

polyrandon Ludic: New framework for Python with seamless Htmx support

I think these kinds of tools are awesome, although I normally prefer not having to rely on getting the functions/classes/types defined for me, I usually find it easier to just use lxml[0] to build HTML components from composable Python functions[1], similar to React.

[0]: https://lxml.de/lxmlhtml.html#creating-html-with-the-e-facto...

[1]: https://ricardoanderegg.com/posts/python-build-html-componen...

polyrandon Show HN: Goqite, a persistent message queue Go library built on SQLite

I also built something similar for Python [0]. I decided to go with a separate DB file for the queue, I think it's better for performance but also to avoid mixing the data storage with the message queue (in my case, I assume the message queue is just an implementation detail and not really tied to the rest of the DB schema).

[0]: https://github.com/litements/litequeue

polyrandon Stanchion: Column-oriented tables in SQLite

No problem, I totally understand! Thanks for the clarification.

polyrandon Should you split that file?

I kind of agree with the post. For me, this technique almost starts overlapping with literate programming[0].

One trick I use[1] is adding certain characters before the section name. e.g:

  ¡¡ settings

This makes searching and jumping between sections a lot easier.

[0]: https://en.wikipedia.org/wiki/Literate_programming

[1]: https://ricardoanderegg.com/posts/write-apps-in-single-file/

polyrandon Llamafile lets you distribute and run LLMs with a single file

The technical details in the README are quite an interesting read:

https://github.com/mozilla-Ocho/llamafile#technical-details

polyrandon Show HN: Pypipe – A Python command-line tool for pipeline processing

Yes, I've used shiv and PEX in the past, and I love those. But that means "adding a build step". Also, as far as I understand, by default, those tools normally decompress the generated zip file on startup, but the decompressed artefacts have to be manually cleaned.

I think they are worth for more complex apps (I actually have a draft blog post with some experiments I did using PEX + gunicorn), but for my use case, it's not worth the effort when I only need the standard library.

polyrandon Show HN: Pypipe – A Python command-line tool for pipeline processing

A bit unrelated, but one thing I absolutely love is the fact you can install it by copying a single file to a folder in your PATH. I have been trying to follow this approach for my Python scripts (standard library only, everything in one file) and I really enjoy the experience. Most of the features I need only require Python 3.8, and Ubuntu comes with Python pre-installed, so

  rsync
  chmod +x

"just works".

polyrandon Choose Postgres queue technology

Some time ago, I wrote a queue using SQLite[0]. Instead of SKIP LOCKED, you can use RETURNING to lock-and-read a message and ensure only one worker is going to pick it up:

  UPDATE ... SET status = 'locked' ... RETURNING message_id

Or you can just use an IMMEDIATE transaction, SELECT the next message ID to retrieve, and UPDATE the row.

On top of that, if you want to be extra safe, you can do:

  UPDATE Queue SET status = 'locked' WHERE status = 'ready' AND message_id = '....'

To make sure you that the message you are trying to retrieve hasn't been locked already by another worker.

[0]: https://github.com/litements/litequeue/

[1]: https://github.com/litements/litequeue/blob/3fece7aa9e9a31e4...

polyrandon Bitwarden: Free, open-source password manager

I really like Bitwarden, but I found it buggy in terms of user experience, and I'm now considering switching. Some annoying bugs I've experienced:

* Desktop app not focusing correctly when opening it

* Browser extension asking to save password even if it's already there

* Log-In suggestions not showing

* Slow and laggy on iOS

* Biometrics log-in breaks half of the times I try to use it

I like the fact it's open-source, although that doesn't make it more secure, and I don't think I would like to self-host.

polyrandon Bun v0.8

PR link: https://github.com/TechEmpower/FrameworkBenchmarks/pull/8337

polyrandon Candle: Torch Replacement in Rust

Interesting. The repo was owned by Laurent Mazare[0] a few days ago, I'm wondering if HuggingFace just hired them to work on candle full time?

[0]: Also maintainer of https://github.com/LaurentMazare/tch-rs

polyrandon How Turborepo is porting from Go to Rust

One detail I enjoy from this post is that sometimes you can just call a CLI[0]. It's easy to spend a lot of time figuring out how to expose some Rust/C code as a library for your language, but I like the simplicity of just compiling, shipping the binary and then calling it as a subprocess.

Yes, there's overhead in starting a new process to "just call a function", but I think this approach is still underutilized.

[0]: https://github.com/vercel/turbo/blob/c0ee0dea7388d1081512c93...

polyrandon Firejail: Light, featureful and zero-dependency security sandbox for Linux

You can now use those on Linux too:

https://justine.lol/pledge/

polyrandon Cloud Backed SQLite

True, but I think there aren't many reasons to not run in WAL mode by default.

polyrandon Cloud Backed SQLite

The .dump command creates a read transaction[0], so it shouldn't block any writes.

From the SQLite docs:

  When a SAVEPOINT is the outer-most savepoint and it is not within a BEGIN...COMMIT then the behavior is the same as BEGIN DEFERRED TRANSACTION

Internally, the .dump command only runs SELECT queries.

[0]: https://github.com/sqlite/sqlite/blob/3748b7329f5cdbab0dc486...

polyrandon Ask HN: Could you share your personal blog here?

https://ricardoanderegg.com/posts/

A mix of tutorials and interesting things I learn or discover. Most of the posts involve Python, SQLite or both.

Some of my favourite posts.

- Extending SQLite with Rust. The basics of writing a SQLite extension in Rust.

https://ricardoanderegg.com/posts/extending-sqlite-with-rust...

- Building a remote SQLite explorer. TL;DR: SSH tricks to turn SQLite into a networked DB.

https://ricardoanderegg.com/posts/sqlite-remote-explorer-gui...

- Using SQLite for logging and ad-hoc profiling (SQLite is a powerful JSON database!)

https://ricardoanderegg.com/posts/sqlite-logging-profiling-p...

- Learning about Bloom Filters by creating one. There are a ton of posts about implementing a bloom filter, but it was super fun to write one from scratch.

https://ricardoanderegg.com/posts/understanding-bloom-filter...

And my favourite one:

- Heroku-style deployments with Docker and git tags. I tried to create a deployment system where I could just `git push production`. It covers a bit of everything, git, docker, Caddy, blue-green deployments, git hooks, curl, etc.

https://ricardoanderegg.com/posts/git-push-deployments-docke...

polyrandon CLI tools hidden in the Python standard library

Yes, I meant the DLL/.so file. The library is almost always present in the OS, the CLI is not.

polyrandon CLI tools hidden in the Python standard library

Python 3.12 will include a SQLite CLI/REPL in the standard library too[0][1]. This is useful because most operating systems have sqlite3 and python3, but are missing the SQLite CLI.

[0]: https://github.com/python/cpython/blob/3fb7c608e5764559a718c...

[1]: https://docs.python.org/3.12/library/sqlite3.html#command-li...

polyrandon Redbean Systems

Really excited to see where this goes. Some time ago, @jart answered a message about using pledge() as WASM alternative [0], which made me believe this could be an alternative to using WASM when you just need a sandbox. I would love to be able to run arbitrary code without having to deal with containers, microVMs or having to use a language that supports WASM [2]

[0]: https://news.ycombinator.com/item?id=34647121

[2]: Even for the languages that support WASM, you usually have to deal with incompatible dependencies, properly configuring your compiler, etc.

polyrandon Makefile Tricks for Python Projects

100% agree, and this bothers me a lot. I like `just`, but for some reason I keep coming back to `make`.

> or not GNU flavor not sure

On my macOS laptop

  which make
  # /usr/bin/make

  /usr/bin/make --version
  # GNU Make 3.81

> doesn’t support parallel tasks by default

I haven't verified this. I've checked `make --help` using the built-in make on my Mac (GNU Make 3.81), it mentions the `--jobs` option to run multiple jobs in parallel.

polyrandon Makefile Tricks for Python Projects

Author here.

I agree my setup is not simple (or even recommended as the default), that's why I call them "tricks".

> you can wire up your imports and script entry points in poetry, and manage venvs there too. Or just use pyenv-virtualenv if you’re on MacOS to auto-use your per-project venv.

As someone who has tried most of the current Python tools (including poetry and pyenv), I wouldn't say that "wiring up" imports and entry points is much simpler or footgun-free. It also requires using specific tools, which may not be possible depending on the project.

Also, multiple venvs is not equivalent, since you still need to manage those venvs and install the packages (+ dependencies). The `PYTHONPATH` trick can save a lot of time when the dependency tree is complex, and you just want a single venv.

For example, a workflow like:

  git worktree add -b feature_one
  #
  # edit some files
  #
  BRANCH=feature_one make run-service
  
  # in a different terminal, assuming `master` is also checked-out as a worktree
  BRANCH=master make run-service

With the `PYTHONPATH` trick, you can do this in a few seconds. Now you have the same service running two different versions of your package, without having to reinstall anything or caring about using the correct venv.

Again, I know this is a hack and it can be very project-specific. But it has worked great for me, and maybe it can be helpful to others.

polyrandon Makefile Tricks for Python Projects

Author here. I like `just` too, but it's missing a few features:

* It can't run tasks in parallel

* It doesn't handle file dependencies. i.e: it's just a task runner, not a build tool.

* It's another tool you have to install, make comes pre-installed in most UNIX-like operating systems.

polyrandon Makefile Tricks for Python Projects

Hi! Author here.

I've looked into pydoit multiple times and I would love to try it. There are a few reasons that make me go back to make (no pun intended!).

* It comes pre-installed on most UNIX operating systems. Yes, macOS ships an older version, but it works in most cases. This makes bootstrapping projects easier. * I like how simple it's to get started with a Makefile when you just need a few tasks. Although I admit, big Makefiles can become hard to maintain.

I've recently experimented with using Makefiles as an ETL orchestrator, and it's becoming quite messy. I believe pydoit is the perfect replacement candidate. I just haven't had time to try it.

Edit:

Another thing I like about Makefiles is that you can keep the "hackiness" of shell scripts, manipulate the environment [0], etc. I guess you can do the same with other tools, but I find it easier to reason about those useful "hacks" in Makefiles.

[0]: https://ricardoanderegg.com/posts/makefile-python-project-tr...

polyrandon Ask HN: What tools would make your Kubernetes development experience better?

I was tinkering with a similar approach to this, using Python objects to generate the YAML. I'm not working in it right now, but I'd love to see other tools use this approach.

polyrandon DuckDB 0.8

Another cool new feature that's not mentioned in the blog post is function chaining:

https://github.com/duckdb/duckdb/pull/6725

I've been using DuckDB for filtering and post-processing data, specially strings, and this will make writing complex queries easier. By combining nested functions[0] and text functions[1], sometimes I don't even need to go into a Python notebook.

[0]: https://duckdb.org/docs/sql/functions/nested

[1]: https://duckdb.org/docs/sql/functions/char

polyrandon Use databases without putting domain logic in them

I have mixed feelings about using stored procedures and triggers. I was once working on a Postgres app and started storing some logic in the DB. I had some triggers that would automatically set values when a user state changed (e.g: when the user changed to 'inactive', the trigger would also update the tables related to subscriptions, API keys, etc).

As another "performance" trick, I was using multiple CTEs with the `RETURNING` clause to execute multiple operations in a single query.

Everything was OK when I was working on that app daily. But then I stopped working on it for a few months, and when I came back, I regretted using those tricks. For example, now I need to verify the triggers to make sure that changing a value won't change other tables that I forgot about. Also, I can't compose the SQL queries I wrote because each query does "everything at once". I would have rather paid the cost of doing 3 queries, and in exchange I could have reused some of those queries in different parts of the application [^1].

Of course, the app didn't even get close to the scale at which 1 query vs. 3 queries matter.

I still appreciate and like having some business logic in the DB, specially `CHECK` constraints. But the tooling for regular programming languages makes everything easier. Having the logic in the DB is a double-edged sword.

[^1]: This can become relevant when building an admin interface/CLI, since you may want to execute partial changes vs. the "everything at once" changes in the user-facing application.

polyrandon SQLite performance tuning: concurrent reads, multiple GBs and 100k SELECTs/s

wal2 mode is not part of the main development branch of SQLite, you need to compile SQLite yourself from that branch. Same with BEGIN CONCURRENT [0]

[0]: https://www.sqlite.org/src/doc/begin-concurrent/doc/begin_co...

polyrandon Experimenting with LLMs to Research, Reflect, and Plan

This sounds cool! I would be happy to read a blog post about it if you ever write one.

polyrandon Experimenting with LLMs to Research, Reflect, and Plan

I've been thinking about using BM25 as a retrieval method to enhance LLMs, I'm happy to see it mentioned here. It can complement vector search, but if it turns out to be useful on its own (maybe with query expansion and other tricks), it could be used as an alternative to vector search when running locally, or in environments with lower compute resources.

polyrandon SQL Maxis: Why We Ditched RabbitMQ and Replaced It with a Postgres Queue

SQLite is missing some features like `SELECT FOR UPDATE`, but you can work around some issues with a few extra queries. I wrote litequeue[0] with this specific purpose. I haven't been able to use it a lot, so I don't have real-world numbers of how it scales, but the scaling limits depend on how fast you can insert into the database.

[0]: https://github.com/litements/litequeue