Absolutely the best font I’ve ever used. I can easily read letters from far, from close, in bright light, in darkness.. used it since day one. Also using it in my terminal. Thanks JetBrains, I love it.
2,352 karma · joined April 23, 2011
https://github.com/19h
kenan [at] sly.mn ksulayman [at] hex-rays.com
Our training dataset is composed of 12 different languages. The English corpus has a size of around 25TB -- ~18TB after a rather aggressive filtering process (char frequencies [i.e. https://gist.github.com/19h/b983769b7ec9c4a9528377d7819892a7], too many unusual features in the sentences, too short sentences, etc.).
Most common misspellings of words have nearly identical fingerprints, and we align the retinas across languages so that translated sentences result in similar fingerprints (via competitive learning).
The 92% is an average and not the exact accuracy across all categories; the accuracy varies by category as every category is represented by its own filter.
We may only be using 16k vector values but that doesn’t mean we only have a vocab of 16k —- our vocab is more around 1.9 million words each described by a sparse fingerprint of 16k.
That said, here's some reading material on the underlying ideas:
- https://en.wikipedia.org/wiki/Semantic_folding
- https://arxiv.org/pdf/1511.08855.pdf ("Semantic Folding Theory And its Application in Semantic Fingerprinting")
This is _not_ TF-IDF. Once you have built the "relation fingerprints" of each word, the fingerprint lookup complexity is o(1) as you'll essentially only load a massive LUT of type HashMap<String, Vec<u16>> (or u32 if you go above 255*255). [pro tip: our LUT has the type HashMap<Vec<String>, Vec<u16>> as our impl also considers bigrams, trigrams, quadgrams]
Unfortunately I can't get extremely specific, but we're also feeding these [u8; 16348] vecs into an HTM w/ spatial pooler; feeding one word-SDR aka fingerprint into the HTM at a time allows you to leverage the HTM to make predictions for the most likely next word-SDR aka the fingerprint of the next word -- if you generalise this on a sentence level, you can use the cosine distance between the actual text-SDR aka fingerprint of the next sentence and the predicted text-SDR out of the HTM to semantically segment paragraphs in a continuous stream of text.
This allows us to segment SOCMINT user2user conversations into individual semantically connected packages of text / messages that can be marked by scenario-specific heuristics to be additionally analysed by a downstream system.
How it works — imagine you’re having these sentences:
“Acorn is a tree” and “acorn is an app”
You essentially keep record of all word to word relations internal to a sentence:
- acorn: is, a, an, app, tree Etc.
Now you repeat this for a few gigabytes of text. You’ll end up with a huge map of “word connections”.
You now take the top X words that other words connect to (I.e. 16384). Then you create a vector of 16384 connections, where each word is encoded as 1,0,1,0,1,0,0,0, … (1 is the most connected to word, 0 the second, etc. 1 indicates “is connected” and 0 indicates “no such connection).
You’ll end up with a vector that has a lot of zeroes — you can now sparsify it (I.e. store only the positions of the ones).
You essentially have fingerprints now — what you can do now is to generate fingerprints of entire sentences, paragraphs and texts. Remove the fingerprints of the most common words like “is”, “in”, “a”, “the” etc. and you’ll have a “semantic fingerprint”. Now if you take a lot of example texts and generate fingerprints off it, you can end up with a very small amount of “indices” like maybe 10 numbers that are enough to very reliably identify texts of a specific topic.
Sorry, couldn’t be too specific as I’m on the go - if you’re interested drop me a mail.
We’re using this to categorize literally tens of gigabytes per second with 92% precision into more than 72 categories.
Worst case you can enable clippy to run in the background.
For instance, obtaining JSON structured data from freetext is a rather impossible task.
That said, for summarization of leq 2k token texts the model performs extremely well.
This is mostly due to its unfiltered nature, where there would be clear biases visible in models like falcon-40b-instruct or other LLaMA derivatives.
For instance, onnx & PyTorch just died randomly and a few thousand dollars later we figured it was directly caused by incorrect assumptions made in the underlying implementations where the developers almost definitely did not have access to an H100 yet.
Depending on how these numbers are obtained, there is a non-zero chance at least part of this increase is caused by us.. note that this number is not indicative of the amount of users or origins (i.e. physical source addresses), but only count directory requests.
I’d absolutely donate on a monthly basis.
In Germany, a termination is valid IFF the termination has been submitted to the employee in writing in a timely manner — that is, they need to have been able to receive it before end of month or it’ll delay the notice period by a month. The standard notice period is 3 months. (I had to do this quite a few times recently unfortunately).
Are worker rights really this bad in the US?
This kind of watermark makes images bad for everyone, not just the AI. Do you really want to see this on every website you visit?
Comparing this to Google Books is silly. Google stole copyrighted books. Copilot uses freely shared open source code. No copyright issue.
The article claims "Open source code on GitHub might be thought of as 'open and freely accessible' but it is not." Lol what? The MIT and Apache licenses explicitly allow reuse. Copilot can absolutely use open source data.
This is typical hype and FUD. No evidence Copilot even used all of GitHub's data or violated any licenses. Baseless speculation.
There's no real antitrust argument here. Nothing to see, move along. yawn