In theory quantized weights of smaller models are under a gigabyte.
If you are looking for megabytes, yeah, those "chat" llms are pretty unusable at that size.
If you are looking for megabytes, yeah, those "chat" llms are pretty unusable at that size.
No comments yet.