HNHacker News
TopNewBestAskShowJobs

2099miles

107 karma · joined December 4, 2023

submissionscomments
2099miles··on DBRX: A new open LLM
Source? I’ve seen this anecdotally and heard it, but is there a paper you’re referencing?
2099miles··on DBRX: A new open LLM
The LLM itself should realize it’s too big and only put the important parts on the gpu. If you’re asking questions about literature there’s no need to have all the params on the gpu, just tell it to put only the ones for literature on there.
2099miles··on Ask HN: Code Llama 70B on a dedicated server
Do you guys really get that much help from these local LLMs? Chatgpt has SO much more functionality and it still is quite limited. Why is it worth it for you guys to run these things locally like what are they doing that provides you so much value?
2099miles··on Ask HN: Code Llama 70B on a dedicated server
That doesn’t quite work.

b. Redistribution and Use.

i. If you distribute or make the Llama Materials, or any derivative works thereof, available to a third party, you shall provide a copy of this Agreement to such third party.

It would be equivalent to you buying something someone else stole aka still illegal. If you wanna use llama you have to agree to their terms as far as legality goes.

As people modify a model to be further and further from the original I’m unsure how the legality and detectability work.

2099miles··on Ask HN: Code Llama 70B on a dedicated server
I run llama chat 70b on a p3 8x large (4 Tesla) and it runs at like 1-5 tokens per sec. And I’m running the model with only 4 bit precision. Are you doing anything else?
2099miles··on Show HN: AI dub tool I made to watch foreign language videos with my 7-year-old
Talked about this idea last month since astrobiology still isn’t all dubbed to English. Thank you for actually making the tool, it’s awesome, huge Kudos.
2099miles··on Problem solving across 100,633 lines of code – Gemini 1.5 Pro Demo [video]
The prompts they used were also different than the ones given like “is this the right order” was “is this the right order, consider the distance from the sun” they put this in their post on Google dev blog.

This one seems to be super straightforward about timeliness and capabilities, but the examples might be a bit simpler than people think. This is pretty amazing but like someone else said you could achieve similar results from rag due to the lack of novelty in these questions and the fact that each dealt with pretty independent examples as opposed to using custom code developed elsewhere in the codebase.

2099miles··on Ask HN: What local machines are people using to train LLMs?
Can you talk about the attention visualization a bit. What’s the benefit. I have been plotting attention masks and even visualizing them over sentences but it obv doesn’t result in a ton of insight due to the sentences changing from words to much more complex vectors that represent those words after being processed through attentions 32x.
← PreviousPage 2 of 2