michael-0acf4

This is similar in spirit to Deep Sets (learning embeddings over unordered collections), but instead of a permutation-invariant MLP it uses a small Transformer encoder without positional encoding. In practice this helps capture things like spelling variation and co-occurrence structure between the tags since they are usually way more well-behaved than generic sets.

The approach itself is very general; however, the available models were trained on imageboard and anime-focused tags. I did some experiments with the clearly biased models on completely nonsensical tags, and they still performed fairly well (suggesting some latent understanding of permutation invariance, cosine scores decrease but not significantly when the one term is a subset of the other).

More links: - https://huggingface.co/michael-0acf4/anitag2vec - https://blog.afmichael.dev/posts/2026/set-embeddings-and-ani...

I think one hot/multi hot encoding might have worked if lexicographic inconsistency is the issue.

Labeling can be done by asking the LLMs to label each tweet from a predefined set of labels, the order doesn’t matter. We can generate these labels either manually or by sampling a small subset of tweets and asking the LLM to tag them (we could be using a word dictionary too). From these labels, we can form a Label-length vector whose entries are sorted alphabetically (ordering is arbitrary but must be consistent).

To populate it, we ask the LLM to tag each tweet by restricting its JSON output to enums composed of our labels. From that, we can form hot vectors, effectively embedding and tagging each tweet at the same time.

We get tags for "free", and as a byproduct, we also get a vector embedding that can be further refined using techniques such as PCA, then you can cosim as usual. The main cost resides mainly on asking the LLM to label each tweet from the predefined set of tags, you can also micro-optimize that by batching prompts ig.

Yes! Good point.

I've been using Syncthing for a while too, but my main issue was that it's really hard to express a network of nodes with it. Both locally (on the same machine) and across the network. For example, setting up a topology like A <--> B --> C <--> D ends up being more painful than it should be while allowing some nodes to be on the same machine.

I've been reading about consensus algorithms too, which got me interested in tackling this problem myself and try making it as simple as possible mainly out of curiosity. Also, this project is designed to be storage-agnostic from the get-go (s3 is on the way).

This is my attempt at creating a prototype of a decentralized store-agnostic file system. More importantly, it runs over HTTP, making it very easy to deploy.

I've been using it for a few days now, mainly for backups and easy access across some of my folders (Video Game Saves, Pictures, ..) on my local network.

It is very similar to a peer-to-peer sharing system, but you almost never know exactly which user created which data. It is simply there. It enables file sharing across a network of nodes, each node will simply synchronize its state from a peer.