llm_trw

Units tests save lives.

I shudder to ask, but what exactly is a math pet?

There are multiple providers for it since it's open source.

Having tried using it, it is much worse than r1. Both the standard and high effort version.

The cookbook is a work in progress by the looks of it.

This is standard notation that's been used for decades.

https://arxiv.org/abs/2402.01790v1

Can someone explain what the point of ollama is?

Every time I look at it, it seems like it's a worse llama.cpp that removes options to make things "easier".

I know a number of people who were building AI systems that either completely dropped off the face of the earth or show up once every few months with an update then disappear again. Granted the majority of them are perverts whose main use case is mixed reality wifus, but still.

I can't exactly blame them.

In the latest furor over deepseek r1 the conversation online was _substantially_ worse than what you'd get from feeding the original paper into r1 and talking to it.

This was the first time where I genuinely wondered what the point of reading groups, message boards and similar is. A model that you can run locally for $6,000 at 20tokens/s beat however many thousands of people because it actually spent the time to read what it talked about.

>HudZah was told that he could be killed by the high voltage, X-ray radiation and possibly other things. This only made him more excited. “My whole intention was, ‘If I fuck up, I’m dead, and this is why I should do it,’” he said.

It sounds like he got exactly what he wanted to.

>I must admit, though, that the thing that scared me most about HudZah was that he seemed to be living in a different technological universe than I was. If the previous generation were digital natives, HudZah was an AI native.

The one thing I'm somewhat ambivalent about is that LLMs are extremely atomizing. It's incredibly easy to go offline for months on end when you're talking to your computer.

If I got hit by a bus in 2005 when I was contributing to Linux and Postgres there were people who would pick up what I was doing and carry it forward.

If I get hit by a bus today, unless someone went through my chats, no one would really have any idea what I'd been working on and carry it on. I have a suspicion that a ton of the best and brightest have gone dark for this reason in the last two years.

Yes, but the r1 model is shiny and new. I've been using it locally for a day and I'm already finding odd spots. One thing that I've seen it do a number of times is get the correct answer in the thinking tokens, then ignore it, generate thousands more tokens until it reaches it context length then give a wrong answer.

There are things, like doc strings for functions, that llama models give better results for.

There is nothing in the deepseek paper that suggests you can't use the order of magnitude in hardware costs you saved to just train models that are ten times as large.

>This is all ignoring the fact that if the h800 was good enough amd's top line cards should be too.

You can't pay me to use AMD cards. I have work to do. I don't have time to write drivers for cards that no one supports.

The point of the deepseek r1 paper is that the more synthetic data you generate by over trained models the better the resulting model. That means more GPUs. There is no reason why you can't train a dozen models like their nameless logic model in other domains and then use them for synthetic data augmentation.

I can't wait for the 2000 dot com crash for AI.

What gets me about the deepseek work is that it's neither novel or theoretically difficult.

I've been using a similar synthetic data pipeline to generate examples for sota bert embedding models.

That low-hanging fruits like this are left unexplored by meta or alphabet - and if we're to be believed lead to crisis meetings for a week straight - makes one wonder wtf the engineers there are doing?

Having read the paper from deepseek I seriously don't get how anyone can think "Oh now is the time to sell nvidia".

Both at the tactical level and strategic level that paper means we will see _more_ nvidia gpus fly off the shelves. If anything it's showing why the h800 is a decent card and everyone in China should buy one (dozen thousands).

>Pony Stark is absolutely a Nazi

>Your Klansplaining is ridiculous

>wtf is wrong with you!

Ah blue sky, keeps reminding me why I left a day after I signed up.

Did the introduction of assemblers lead to creating more or fewer programming jobs?

The only people who think that Ai models won't result in more demand for human labour are the ones who have never used them.

End result is on par with o1 preview, which is ironically more intelligent than o1, but the intermediate tokens are actually useful. I've got it running locally last night and out of 50 questions so far I've gotten the answer in the chain of thought in more than half.

Jesus we are reaching levels of blinking for torture of these models: https://www.youtube.com/watch?v=WZ256UU8xJ0

It's hardly more brute force than using a trillion parameter model in the first place.

The distils run on the same hardware as the llama models they are based on llama models anyway.

The full version... If you have to ask you can't afford it.

>There is still no mechanism in GenAI that enforces deductive constraints (and compositionality), ie., situations where when one output (, input) is obtained the search space for future outputs is necessarily constrained (and where such constraints compose).

I build these things for a living.

This is a solved problem.

You use multiple different types of models to supervise the worker models and force them to redo the work until you get a result that makes sense, or they fail and you give the resulting dump to a human to figure out what went wrong or ignore it.

Inference time compute is through the roof, but when you can save thousands of dollars by spending hundreds it's a no brainer.

Some people want AI to be as infallible as god before they'd consider it useful.

An rag model can already sort your email.

Its just that it costs too much to do that for the hoi polloi who think everything digital should be free forever.

Llama is worse than gpt4 because they are releasing models 1/50th to 1/5th the size.

R1 is a 650b monster no one can run locally.

This is like complaining an electric bike only goes up to 80km/h

I use qubes.

A corporate antivirus is a rather large step down.

It's because people now treat parties like they used to treat football teams.

ML workloads where I can do a few epochs locally to see where the data is going wrong before running the full training run in the data center.

Ones without a straitjacket of bureaucracy.

It's not like I can't exfiltrate all their data with a python script and a web cam.

The hardware costs tens of billions. The electricity cost is around what you guessed.