This seems redundant and requiring more overhead while providing less reliability than something like Kiwix to me.
That way the tiny model isn't acting as a potentially incorrect interpreter but you still get semantic search to help you find the right documents quickly.
I suppose there is something to be said for a good summarization, I just worry that these tiny models aren't really GOOD summarizers.
I'm all up for using the better/simpler task for the job. That's been a thing lately also with the Jev/decision models hype, most of those things can either already be done by a pretrained classifier[0], or you can start with a bigger model and then train your smaller/adhoc one when you have some data (it doesn't even take much for most applications, you can get about 95% accuracy on email classification with 50-100 examples)
0: http://jeffyclassify.com/blog/when-llms-fail-at-classificati...
The bigger issue in this context would be canning, such as incorrectly explaining the mechanisms to avoid botulism (acid and/or pressure canning), or perhaps incorrectly wiring in repairs. However, again, it’s best to practice with these prior to emergencies anyway.
It should be noted though, that the model does source its claims, so you can fact check it and should review this information regularly.
Consider they won't have access to the plentiful practice opportunities we have now, most resources would have to be spent with a practical goal.
A field guide to foraging for edible mushrooms will probably be more practical than naïve use of an LLM: “can I eat this mushroom?” is an exercise in trust that I will maybe never be comfortable with.
But “how do I effectively forage for edible mushrooms” is a great stepping stone.
One of these situations prohibits the other.
If things are so bad that you need to factor in the non-operability of any electronic devices, how many indestructible paper libraries are you planning to build and manage in advance, and at what budget?
If there's any place to get an answer it's here.
Bravo my friend. And I mean that in a good way. Love me some diversity.
Either way the comparisons here are unfair. An LLM that runs off a backpack-sized solar panel and a laptop might be a substitute for a broad, low quality, surface-level encyclopedia but it's many orders of magnitude worse than a library of books.
I would argue that in a survival situation the accuracy of information becomes even more important. If an LLM to misidentifies a plant (or a medical condition, its treatment, etc.) it could put you in an ER, but in a survival situation that could kill you.
That seems to have been mostly solved through scale. I was asking Opus 5.5 about some obscure stuff and it kept hedging "this is from memory since I don't have internet access" (it's sandboxed), "I'm reasonably sure about this but not about the numbers" etc.
Even though it was obscure enough that Google couldn't easily find the paper that Opus remembered (I couldn't find it at all with the traditional search results; the AI summary found it on my third try), it actually didn't hallucinate it.
In fact I can't remember Opus 5.5 ever hallucinating. They've clearly done some work on that.
Obviously this local LLM is going to hallucinate all over you though.
This is more for elaborate projects where you don't know precisely what you are looking for. Questions I can think of in a survival scenario:
- my solar panel connector is broken. We only have 8 hr battery left. Can you tell me how to DIY repair it with the following parts
- we want to receive emergency FM broadcasts from the state. Give me steps on how to receive it on Flipper Zero with this breakout board I found in my spare parts
- with the dewalt drill and these parts, can you tell me how to build an outboard motor for the canoe. We need to get outta here before the flood goes up further
Kiwix is great, but I'd also like a language model to be able to deeply interrogate all of that information much faster than I can read.
Just arrange its harness so that its responses are grounded in the source material and require source citations for everything. Then read the citations to confirm it's not a hallucination.
Except the build process there requires a specific recent version of swift. Hardly a good definition of "anything"
You can run bonsai2 27b on basically any gaming GPU from the last 15 years including on game consoles.
Everyone has access to this stuff now if they bothered to look beyond the marketing from big labs trying to collect subscription revenue.
I work on compilers, kernels, and operating systems using exclusively consumer GPUs.
Undoubtedly useful in the right contexts, particularly for local repair or homesteading, but “survival” does imply that this would be more useful in an SHTF situation.