Acumen321

Quantization in this context is the precision of each value in the vector or matrix/tensor.

If the model in question has a token embedding length of 1024, even if it was a 1 bit quantization, each token has 2^1024 possible values.

If the context length is 32,000 tokens, there are 32,000^2^1024 possible inputs.

Very cool. If you like this you will like ZenTrails on Steam also.

This is my favorite story of his. It is part of the collection Axiomatic for those new to him.

Anyone who would be on this site is the kind of person who should do themselves a favor and read his short story collection Axiomatic. His other collections are excellent as well, I have read them all. For novels, Diaspora is my favorite so far, but I am still working through them.

Mike Acton's talk "Data-Oriented Design and C++" from CppCon 2014 is the best programming talk ever given in my opinion. A must watch:

https://youtu.be/rX0ItVEVjHc

But maybe you get a few more seconds, one more swing, and take out your opponent. Your tribe kills the lion or wins the war. If the initial mutation wasn't detrimental so wasn't tested right away on an individual basis, but allowed to reproduced to form a population, it would be the group test that gets passed. Just a thought.

A theory I heard that I like, is that if you are getting attacked from the right, and suffer damage to the right side of your head, you probably want your right arm to work more than your left for the highest chance of fighting off the attack.

You start a bank, I come to take out a loan and I want it in cash since I am buying a used car this afternoon. You have no cash since you said you don't need it, are just going to create it. Where does the cash come from? If you get it somewhere else, like the FED, you clearly aren't creating it, the FED is, right?

If you need to involve the FED (which you don't as per how I described) then the FED creates the money, not the banks. This invalidates your entire premise.

It is (was) a thing, the idea works exactly the same way. Person A deposits $100, bank lends $90 to B, $10 goes to reserve (if 10% reserve rate). System thinks there is $190 instead of $100, so money is "created".

As of 2020 in the US the reserve rate is 0%: https://www.federalreserve.gov/monetarypolicy/reservereq.htm

There are many problems with your understanding, but the simplest total failure of your model is that if the bank did just get the money from somewhere else to lend, it is not creating it.

You are not describing the bank "creating" money, which they actually do as per how I described. You are describing the bank borrowing money.

The loan guy absolutely does need to make sure the bank has the cash to make the loan. You can't loan more money than you have. What happens when B goes to withdraw it from the bank to buy a car or house if it isn't there?

If banks can loan more than they have by say borrowing the money at a lower rate than they lend it, that invalidates your basic premise of banks creating money. They wouldn't have created it, they would have borrowed it.

This is a very popular but false 'take the gist of it as true' misunderstanding.

Yes, banks do "create" money. No, it is not out of thin air. It absolutely does come from deposits.

An example of how banks "create" money, is person A has $100. A deposits it. The bank lends that $100 to B. Now B has $100, but A also still thinks they have $100, even though they just have a number on a piece of paper.

They system goes from acting as if $100 exists, to acting as if $200 exists, but really there is only $100, and an IOU for $100.

That is what bank money "creation" is. It is not from thin air.

People get confused because "money", as in fiat currency, is also a Government IOU. But the above principle is true for gold, bitcoin, or any asset, and they wouldn't get mixed up the same way thinking banks create gold out of thin air.

It is a starkly different thing than how the Government creates money.