minxomat

Is this related in any way to the actual ChatLLama project that is developed here: https://github.com/nebuly-ai/nebullvm ?

The former. Makes sense for their business model.

Follow up recommendation: for Spanish, “Madrigals Key” is language transfer in book form. It can in fact be used as a workbook for this course.

Sure. Sent a ping.

We're not far off: https://imgur.com/a/nJxxcUg

A plot point in an episode of Person of Interest, where AGI "Samaritan" funds a charity (free tablets for students, Samaritan access pre-installed) to take over education and recruit more mercenaries.

I'm building this, for (mostly) non-scientific non-fiction works (books, articles, news, etc.). Launching soon, with about 7,500 books indexed.

Generally, what I found useful to build a graph between "topics" or entities was to use a HyDE[1] prompt to generate possible distinct definitions and then build a nearest-neighbor network from that. This successfully identifies related concepts in the truly abstract sense rather than literal entities.

[1] https://summarity.com/hyde

The whole point of the LLaMa paper is that large models are undertrained and oversized.

A laptop and a desktop (Mac Studio)

This has been fixed almost 2 days ago now. It’s literally mentioned at the top of the repo.

The full use case includes quantisation, which the repo points out uses a large amount of system RAM. Of course that’s not required if you skip that step.

You’re missing something. Both SHP (https://huggingface.co/datasets/stanfordnlp/SHP) and OpenAssistant datasets are referenced.

And the TOS violation might be the case, the project nevertheless has a mode to use OpenAI in the fine tuning steps.

With 16 threads, about 140ms per token for 30B, 300ms per token for 65B

I should also mention that 65B should be able to run on 64GB systems. Total system memory consumption on M1 Ultra is about 67GB when running nothing else.

So if I'm reading this right, 65B at 4bit would consume around 20GB of VRAM and ~130GB of system RAM?

There are open datasets (see the chatllama harness project and its references). You can of course also cross train it using actual ChatGPT.

AdGuard

> not really competitive with ChatGPT

That's impossible to judge. LLama is a foundational model. It has received neither instructional fine tuning (davinci-3) nor RLHF (ChatGPT). It cannot be compared to these finetuned models without, well, finetuning.

GPT-3 might be the ultimate contender for https://en.m.wikipedia.org/wiki/Hutter_Prize - now if we could only find the prompts to recover it all

Everything old is new again … https://en.m.wiktionary.org/wiki/juvenoia

Not to forget the Toshiba AC100 with the original Tegra, which was one of the first usable ARM laptops (netbooks?) and helped much of Linux desktop support development for ARM in the early days.

Not really, that’s exactly what Papertrail does.

Would love to use an API, per token pricing is a good approach (with use limits like OpenAI). If you need testers, I have some use case (long form non-fiction content). LMK at ml[at]summarity[dot]com

I second this, API please.

As an interviewee you can use the interviewer's typing as a good signal. When I'm conducting an LP interview and you go off on a tangent that I'm not interested in, I'll stop typing and look up long before I'm likely to interrupt you.

Now I'm not _expecting_ anyone to pick up on that. My point is it's a useful skill to engage (even via zoom) with the other person and develop a feel for how they're processing the info you're giving them. Otherwise the interviewee could just talk to a transcription AI in the first place and the hiring team reads the summary (shudders).

Though now I've said that, someone is probably going to build this nightmare version.

I’ve done some experiments and the ideal seems to be semantic chunks of max 175 tokens. But only if those chunks are already very dense (use summarisation to get there)

That’s commonly referred to as abstractive vs extractive summarisation

Precedent would be Blinkist, which AFAIK just uses an army of editors. Though, they probably _have_ contracts with the publishers

> kind of cohesive conceptual understanding of a single object

It doesn't, it just requires character sheets in the training set, the rest is basically a style transfer.

Rigging has come a long way with AccuRig and Cascadeur in 2022, approaching diminishing returns.

Not really. I see fly competing with Workers for smaller services, and with KV and D1, managed edge DBs are so easy it’s hard to live without them. Very different use case from a traditional managed DB - because it’s either all scaling and running in fly or it isn’t.