863 karma · joined January 7, 2015
In my first few tests, it was pretty meh (not as good as Vercel's V0) but then I figured out how to change the model away from GPT 3.5 and it got a lot better. I also appreciate making the system prompt etc. configurable!
Suggestion: Using placeholder images whenever an image/icon is included would be nice.
In general, you'd have an actual database and make it so users can only see their own data! See https://github.com/AnswerDotAI/fasthtml/blob/main/examples/a... which adds a filter to queries and DDL statements to ensure that the user can only see/edit their own todos.
- Incremental complexity - starts super simple and I can add stuff as I need it. I don't like frameworks where step 1 already leaves you with lots of files and a bunch of things you need to know.
- Easy escape hatches. I like some of the simpler demo/dashboard things but inevitably hit a ceiling that suddenly requires a lot of hacking to get past. Since FastHTML is a lot more transparent it's very easy to go right in and do something with JS or mess with the request or make something custom. So you're not stuck with only the widgets a framework gives you or anything like that.
- "Parameter Efficient" finetuning methods let you customize LLMs without having to train all the parameters
- But LoRA (the most popular method) didn't match full finetuning performance on some tasks
- DoRA closed the gap while still being very efficient
- Quantization (representing the original weights with fewer bits per parameter) makes things even more memory-efficient
- FSDP lets you spread the work over multiple GPUs, using less memory on each one.
The upshot is that where you previously needed, say, 8 fancy Nvidia A100s to fine-tune an LLM you can now do so on a few 3090s, and while it might take a little longer you're at least getting something almost as good as (or in some cases possible better than) the full finetuning equivalent.
- 3D-printable parts storage solutions (via: I found some part storage bins in the discard pile at a local hackerspace)
- MITM proxy to snoop on Github Copilot API requests (via: we're building an jupyter AI assistant thing and got curious how other players do it).
- DIY robot arms (via: I'm making several for a nested 'you pass butter' joke, via a casual conversation about robotics being accessible now. YouTube is amazing at surfacing smaller makers once you start watching a few videos on a given topic)
- Learning about Oauth and JWT (via: 'why is auth still a pain?')
- Invertebrate UV fluorescence (via: that millipede is glowing under my UV torch!)
(a small subset of these end up documented https://johnowhitaker.dev/all.html eventually if you're curious to see a longer historical list)
I like rabbit holes where following the curiosity gradient to a satisfying conclusion is possible. "How does X work" leads eventually to code that does X. I'm less happy when they lead into a tangle of complexity, like digging into a library only to find weird abstractions 6 layers deep or trying to compare 18 different alternatives in a field I don't know very well.
OP I'd also like to hear yours!
Under more realistic assumptions, the amount of alcohol produced by BCS3-L1 probably isn’t significant even by the very low standards of natural blood alcohol concentrations."
(From https://www.astralcodexten.com/p/defying-cavity-lantern-biow...)
For those talking about breakeven points and cheap cloud compute, you need to factor in the mental difference it makes running a test locally (which feels free) vs setting up a server and knowing you're paying per hour it's running. Even if the cost is low, I do different kinds of experiments knowing I'm not 'wasting money' every minute the GPU sits idle. Once something is working, then sure scaling up on cheap cloud compute makes sense. But it's really, really nice having local compute to get to that state.