rain1

You can offload only 10 layers or so if you want to run on a 4GB card

my understanding is that the engine used (pytorch transformers library) is still faster than llama.cpp with 100% of layers running on the GPU.

exciting! maybe we will see that land in llama.cpp eventually, who knows!

I haven't tried that but https://github.com/abetlen/llama-cpp-python and https://github.com/r2d4/openlm exists

4-bit quantization is to reduce the amount of VRAM required to run the model. You can run it 100% on CPU if you don't have CUDA. I'm not aware of any AMD equivalent yet.

not at all, your question was really good so I added the answer to it to my gist to help everyone else. Sorry for the confusion I created by doing that!

I'm sorry! I added this improvement based on that persons question!

Apologies for that. I've added some extra micromamba setup commands that I should have included before!

I've also added the git clone command, thank you for the feedback

It's just for fun!

These local models aren't as good as Bard or GPT-4.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet.

It is also possible to run fine tuned versions like vicuna with this. I think. Those versions are more focused on answering questions.

no

this is not correct

"I Crashed My Airplane" - TrevorJacob

https://www.youtube.com/watch?v=vbYszLNZxhM

python newbie here, why did messages change from [ to (

There is no point in constructing a fixed template JSON object like that just to parse it again.

thanks for clarifying that "pornhub" is an adult website

> Together with Yann LeCun, and Yoshua Bengio, Hinton won the 2018 Turing Award for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing

Is there a mod that adds PNP/NPN style transistors?

why is it called serverless when it is a server

But then it's an LLM augmented with a tool that is solving the problem, and not the LLM itself.

I've experimented with this problem a bit https://github.com/rain-1/gpt-incontextlearning-experiments/... - the results were that they can kind of do it, but they do go off the rails fairly easily and get wrong answers.

Is there a way to share my transcript?

we do, it's called the algebraic numbers!

every polynomial with algebraic coefficients has 'n' solutions (counted with multiplicity)!

so e.g. x^121 + sqrt(7)x^9 + fithroot(22)x^7 + (1+i)x^3 + 22/7 = 0 has 121 solutions. and they're all algebraic numbers: nothing weird like pi in there.

Nothing much of interest there.

explorer here: https://evanthebouncy.github.io/chatgpt-svg/

I have compiled a list of some of the materials that I found the best for learning about how LLMs work. I Hope this is useful to you all. I will continue to update it as I find new things.

I've included the exact prompt that I gave ChatGPT to get it to answer the questions without saying "As an AI assisant/language model blah blah blah" https://gist.github.com/rain-1/3bf56122b0ebeac929dff0f881ee8...

Do you have any recommendations on where to learn this? Ideally some online course or just a single really good textbook to study?

Oh I think I see what you mean, note that I used two different AIs (GPT-4 was the tester, Me & ChatGPT were the testee's).

That's absolutely right, it just predicts the next token. One of the discoveries that led to GPT was the concept that "token prediction is universal" in the sense that all other natural language tasks are a sub-task of token prediction. For example translating between one language and another is just predicting what would continue after you say something then say "and here it is in french: ".

There are levels to token production from generating complete jibberish, to generating very shallow nonsense sentences, to generating gramatically coherent sentences that dont really say anything, .. and so on. They've pushed the depth of its token prediction to a new level that makes it more impressive than anything previous.