13,114 karma · joined March 24, 2008
The downside is that it makes development and testing super slow without the speedup you get from having local GPU power.
I'm nothing even close to a knowledgeable expert on this, but I've dabbled enough to have an idea how to answer this.
The special thing about why GPU's are used to train models is that their architecture is directly applicable to parallelize the kind of vector math that goes on under the hood when adjusting weights between nodes in all the layers of a neural network.
The weights files (GGUF, etc) are just data describing how the networks need to be built up to be functional. Think compressed ZIP file versus uncompressed text document.
You can run a lot of models on just a cpu, buts its gonna be slooooooow. For example, I've been tweaking on running a Mixtral8x7b model on my 2019 Intel Macbook Pro with llama.cpp. It works, but it may be running at 1-2 tokens per second at max, and that's even with the limited GPU offloading I figured out how to do.
Links to an 11 year old article.
I accidentally dunked my scale in the sink the other day. Luckily it's working now but it wouldn't turn on for a few days. So I was doing pancakes and couldn't precisely measure stuff and they came out terrible, like before I dialed in the precise ratios I use by weight. Scale is working again, so today they came out perfect.
I'm no great cook or baker, but I've learned through trial and lots of error how to bake bread and a few other things. Get a scale and always convert to grams.
I have all the time in the world to read things that are interesting (to me) and really absorb the content. But I've increasingly been noticing a "recipe article" trend with blog posts. People throw 100 words in when 10 will do. This article is an example.
I've even gotten to the point with certain important blog posts where I copy paste the content to chatgpt with a simple prompt to summarize the main points.
I'm not happy that I have to do this, but people have seemingly forgotten the art of getting to the f'ing point.
Your audience has an attention slightly shorter than this comment. Unless you catch their attention, none of your nuance is going to matter.
Generally I've found just skimming the first line of each paragraph works well, but even that is getting harder to find.
Location: San Francisco
Remote: remote/hybrid/in-office
Willing to relocate: no
Technologies: full stack engineer, JavaScript, node.js, Python, html/css, React and other modern web frameworks, recently have been deep diving on LLM's and ML pipelines
Résumé/CV: on request
Email: charles@geuis.com
Github: https://github.com/geuis
LinkedIn: https://linkedin.com/in/geuis SEEKING WORK
Location: San Francisco
Remote: remote/hybrid/in-office
Willing to relocate: no
Technologies: full stack engineer, JavaScript, node.js, Python, html/css, React and other modern web frameworks
Résumé/CV: on request
Email: charles@geuis.com
Github: https://github.com/geuis
LinkedIn: https://linkedin.com/in/geuisSome suggestions:
Touch controls are sensitive. Add some kind of specific control pad area instead of the entire screen. Kept running into the same objects over and over and getting the same dialogs.
Don't show the entire dialogs again and again. Provide some kind of prompt like "Read?"
Hitting "Close" should stop the dialog rendering immediately vs waiting for it to finish every time.
Politeness usually leads the way.
https://www.downpour.com/the-corporation-wars-insurgence?sp=...
I take that back. That domain doesn't resolve and google doesn't provide any direct alternate links. Was it an old service acquired or shut down since the last time you used it?
They also released their API a week or 2 ago. Its significantly faster than anything from OpenAI right now. Mixtral 8x7b operates at around 500 tokens per second. https://groq.com/
Sally has 1 sister.
Here's the logic behind this:
1. We know that Sally has 3 brothers.
2. Then we are told that each brother has 2 sisters.
3. Since Sally is one of the children in the family, there must be another sister besides Sally.
4. Therefore, Sally has 1 sister.
The number of sisters Sally has is not affected by the number of brothers she has, so the information about her brothers having 2 sisters is not relevant to finding the number of sisters Sally has. SEEKING WORK
Location: San Francisco
Remote: remote/hybrid/in-office
Willing to relocate: no
Technologies: full stack engineer, JavaScript, node.js, Python, html/css, React and other modern web frameworks
Résumé/CV: on request
Email: charles@geuis.com
Github: https://github.com/geuis
LinkedIn: https://linkedin.com/in/geuis Location: San Francisco
Remote: remote/hybrid/in-office
Willing to relocate: no
Technologies: full stack engineer, JavaScript, node.js, Python, html/css, React and other modern web frameworks
Résumé/CV: on request
Email: charles@geuis.com
Github: https://github.com/geuis
LinkedIn: https://linkedin.com/in/geuisI think we're already dealing with this. My uncle is in his 60's and maintains old truck shipping software in COBOL. Btw there are job openings in old tech like this, for those that are interested. Happy to provide introductions.
But the basic problem stands, the left-pad issue.
We still deride this choice. Junior engineers without supervision hap-hazardly installing dependencies. But over the course of decades and generations of developers we still "sum to zero", where most software will rely on some number of unknown dependencies.
Say in 2100 an update needs to be issued. You push it through whatever is managing npm dependencies at the time. Meanwhile there is a solar system of dependent devices that need security updates. There could be trillions of dependent devices and any number of of independent intermediary caches that may or may not be recently updated. I can't event imagine what that dependency tree would look like.