jncraton

You are right that the concept of "safe" is nebulous, but the goal here is specifically to be XSS-safe [1]. Elements or properties that could allow scripts to execute are removed. This functionality lives in the user agent and prevents adding unsafe elements to the DOM itself, so it should be easier to get correct than a string-to-string sanitizer. The logic of "is the element currently being added to the DOM a <script>" is fundamentally easier to get right than "does this HTML string include a script tag".

[1] https://developer.mozilla.org/en-US/docs/Web/API/Element/set...

2 and 3 bit is where quality typically starts to really drop off. MXFP4 or another 4-bit quantization is often the sweet spot.

That's great. Here's "me" implementing a JS version of that library in one shot using Github Copilot and a 1 sentence prompt:

> Implement when.js as a simple, zero-dependency js library following SPEC.md exactly.

https://github.com/jncraton/whenwords/pulls

I've adjusted or removed those sentences in the article.

Pyodide supports numpy and scipy.

https://pyodide.org/en/stable/usage/packages-in-pyodide.html

The languagemodels[1] package that I maintain might meet your needs.

My primary use case is education, as myself and others use this for short student projects[2] related to LLMs, but there's nothing preventing this package from being used in other ways. It includes a basic in-process vector store[3].

[1] https://github.com/jncraton/languagemodels

[2] https://www.merlot.org/merlot/viewMaterial.htm?id=773418755

[3] https://github.com/jncraton/languagemodels?tab=readme-ov-fil...

It would be nice to see the Phind Instant weights released under a permissive license. It looks like it could be a useful tool in the local-only code model toolbox.

The speedup would not be that high in practice for folks already using speculative decoding[1]. ANPD is similar but uses a simpler and faster drafting approach. These two enhancements can't be meaningfully stacked. Here's how the paper describes it:

> ANPD dynamically generates draft outputs via an adaptive N-gram module using real-time statistics, after which the drafts are verified by the LLM. This characteristic is exactly the difference between ANPD and the previous speculative decoding methods.

ANPD does provide a more general-purpose solution to drafting that does not require training, loading, and running draft LLMs.

[1] https://github.com/ggerganov/llama.cpp/pull/2926

You might be interested in "Text Embeddings Reveal (Almost) As Much As Text":

> We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes.

https://arxiv.org/pdf/2310.06816.pdf

There's certainly information loss, but there is also a lot of information still present.

Google released the T5 paper about 5 years ago:

https://arxiv.org/abs/1910.10683

This included full model weights along with a detailed description of the dataset, training process, and ablations that led them to that architecture. T5 was state-of-the-art on many benchmarks when it was released, but it was of course quickly eclipsed by GPT-3.

It was common practice from Google (BERT, T5), Meta (BART), OpenAI (GPT1, GPT2) and others to release full training details and model weights. Following GPT-3, it became much more common for labs to not release full details or model weights.

> PNG uses deflate. General byte-level patterns. It does not do bespoke image-specific stuff.

That's not quite the whole story. PNG does include simple filters to represent a line as a difference from the line above, and that may be what the original post is referring to. [1]

[1] https://en.wikipedia.org/wiki/PNG#Filtering

There is a large amount of theoretical research on the subject of energy limits in computing. For example, Landauer's principle states any irreversible change in information requires some amount of dissipated heat, and therefore some energy input [1].

Reversible computing is an attempt to get around this limit by removing irreversible state changes [2].

[1] https://en.wikipedia.org/wiki/Landauer%27s_principle

[2] https://en.wikipedia.org/wiki/Reversible_computing

You might be interested in OpenWorm:

https://openworm.org/

This paper might be helpful for understanding the nervous system in particular:

https://royalsocietypublishing.org/doi/10.1098/rstb.2017.037...

This is great to see. It looks like the size of the embedding vector is half the size of text-embedding-ada-002 (768 vs 1536) while providing competitive performance. This will save space in databases and make lookups somewhat faster.

For those unaware, if 512 tokens of context is sufficient for your use case, there are already many options that outperform text-embedding-ada-002 on common benchmarks:

https://huggingface.co/spaces/mteb/leaderboard

You might be interested in TinyStories:

https://arxiv.org/abs/2305.07759

> In this work, we introduce TinyStories, a synthetic dataset of short stories that only contain words that a typical 3 to 4-year-olds usually understand, generated by GPT-3.5 and GPT-4. We show that TinyStories can be used to train and evaluate LMs that are much smaller than the state-of-the-art models (below 10 million total parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate reasoning capabilities.

OpenLLaMA models up to 13B parameters have now been trained on 1T tokens:

https://github.com/openlm-research/open_llama

Thanks for pointing that out. Classification is half-baked at the moment. It should ultimately be restricting output to only appropriate labels, but right now it is simply sampling.

You can actually get these models to do this, but you have to ask:

    >>> lm.do(f"Answer from the context: What is YCombinator? {lm.get_wiki('Python')}")
    'The context does not provide information about YCombinator.'
    >>> lm.do(f"Answer from the context: What is YCombinator? {lm.get_wiki('YCombinator')}")
    'YCombinator is an American technology startup accelerator that has launched over 4,000 companies, including Airbnb, Coinbase, Cruise, DoorDash, Dropbox, Instacart, Quora, PagerDuty, Reddit, Stripe and Twitch.'

Without being told to be grounded, the model will guess. However, it may be able to identify information not available in a provided context.

One of my goals for this package is to provide a way for folks to learn about the basics of grounding and semantic search.

That's correct. The current base model is an int8 quantization of LaMini-Flan-T5-248M described here:

https://github.com/mbzuai-nlp/lamini-lm

I shared more details over on Reddit:

https://www.reddit.com/r/LocalLLaMA/comments/14btk3a/explore...

I appreciated Richard's take on AI-assisted programming:

> It doesn’t remove the need to communicate precisely. ... In a sense, that’s almost the definition of what makes a programming language a programming language, as opposed to some other kind of language. There’s a precise semantics to everything that is said in that language.

> With the advent of AI-assisted programming, now we have sort of a new method of communication in that it’s a communication from computer back to human. In that, you might have something like ChatGPT producing the code, but a human still has to read that code and make sure that it does what you think it does. And as a medium of precise communication, it’s still very important to have a programming language that allows that communication to happen.

> There’s usually no need to go beyond 16-bit accuracy, and most of the time when you go to 8-bit accuracy there is too much loss of resolution.

I'm not sure this is accurate. From what I have seen, 8-bit quantization is usually fine, and even 4-bit is a viable tradeoff. Here are some benchmarks from TextSynth showing no significant degradation between 16 and 8 bit:

https://textsynth.com/technology.html

8-bit uses half as much memory and doubles the throughput for limited quality loss.

> could camera manufacturers add a new tag to the EXIF data that is cryptographically signed by the camera

They could, and some already do, but this doesn't fully solve the problem. You'd still be left with the analog loophole[1]. A sufficiently advanced attacker could feed a generated image directly to the camera sensor.

[1] https://en.wikipedia.org/wiki/Analog_hole

Embeddings can be trained specifically to cause questions and content including their answers to have similar representations in latent space. This has been used this to create QA retrieval systems. Here's one commonly used example:

https://huggingface.co/sentence-transformers/multi-qa-MiniLM...

What is wrong with the CodeGen model that they are using?

It is a reasonably large model (up to 16B params) that has already been trained on both natural language and code. I would expect it to underperform larger models, including GPT-3.5 and GPT-4, but this should still be very useful for autocomplete and boilerplate in simpler cases. It is a bit under trained compared to Chinchilla, T5, or LLaMA, but it still performs well.

According to the paper[1], this model is competitive with the largest Codex model that was the basis for Copilot originally.

[1] https://arxiv.org/pdf/2203.13474.pdf

This can certainly be done. Here's one example from 2021 demonstrating training an LLM to use a scratchpad ("talking to itself") to greatly improve accuracy on arithmetic problems:

https://arxiv.org/pdf/2112.00114.pdf

If an example would be helpful, OpenAI's tokenizer is publicly usable on their website:

https://platform.openai.com/tokenizer

You can drop sample text in there and visually see how it is split into tokens. The GPT2/3 tokenizer uses about 50k unique tokens that were learned to be an efficient representation of the training data.

OpenAI provides tools to fine-tune models:

https://platform.openai.com/docs/guides/fine-tuning

The largest Davinci model currently costs $0.03 per 1k of training tokens during fine-tuning, so you could fine-tune a model on 10M tokens for one epoch at a cost of $300.

I did something similar in a simple whiteboarding app that I made a while back:

https://github.com/jncraton/box-line-text

The data representation could probably improve in terms of readability, but I'm happy with the way it provides a concise encoding while still including the text from the document directly:

https://box-line-text.netlify.app/#;c2121Hello;5321;c1341Hac...!

This is the process of mapping documents to vectors representing their semantic meaning. Here's a reasonable high-level overview:

https://en.wikipedia.org/wiki/Sentence_embedding

You can use any sort of natural language text as input. The system transforms the input text into a multidimensional vector that roughly represents the semantic meaning of the input.

If you do this for many different inputs, you can get representations for each of them and store them in a database alongside the inputs. From there, you can use traditional methods to search for nearby vectors to efficiently search by semantic meaning.

One earlier related example is word2vec from 2013. This tool transforms individual words into embeddings, but is conceptually similar. Wikipedia has a decent overview that might be helpful:

https://en.wikipedia.org/wiki/Word2vec

That work demonstrated the utility of transforming meaning into vector space for search but also for basic semantic reasoning. For example, you could perform an operation like "brother - man + woman" on the embedding and the result would be an embedding very close to "sister".