HNHacker News
TopNewBestAskShowJobs

brucethemoose2

7,874 karma · joined February 13, 2023

submissionscomments
brucethemoose2··on Still no love for JPEG XL: Browser maker love-in snubs next-gen image format
When the Google JXL controversy first went down, I found that Google's commit rejecting JXL was authored by someone with AOMedia contributions, and that the manager who signed off and commented on it had some interview about the benefits of AV1.

The links are buried somewhere on Phoronix, I am looking... But what I am saying is Google's rejection of JXL seems to be as bad as it looks.

brucethemoose2··on How much bigger could Earth be before rockets wouldn't work?
A civilization just a few decades ahead of us is (theoretically) almost unimaginable. Bio augmentation, true AIs, who knows what advances in fundamental physics knowledge... Just to start.

What I'm saying is that whatever engineering and environmental limitations we currently perceive are probably irrelevant.

brucethemoose2··on How much bigger could Earth be before rockets wouldn't work?
Is this really a big factor?

Not being multiplanetary seems like the least of our existential problems here on Earth, and will continue to be that way for awhile.

At the same time, chemical rocket efficiency becomes totally irrelevant for a slightly more advanced civilization than us.

brucethemoose2··on Mark Cuban Enters Elon Musk's Echoverse of Madness
I like how there's a Musk SEO autoplay video right in the middle of the article.

Even an analytical article on the Musk cult can't escape the Musk cult. Its just too profitable.

brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
Unfortunately no. Falcon is pretty much worse than other models at the same parameter count.
brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
That's pretty much the trick.

Discord is so time inefficient, its almost hilarious. For every incredible conversation between experts you observe, you have to weed through 200 times as much filler.

brucethemoose2··on Meta's Reality Labs loses record $4.65B ahead of Apple's Vision Pro launch
> But engagement is a proxy and I think we need to start actively acknowledging that the metric has been long hacked and at this point abused.

It will not be, as long as it is the internet gatekeepers' livlihood.

brucethemoose2··on Meta's Reality Labs loses record $4.65B ahead of Apple's Vision Pro launch
> Why can I not just get one self contained coherent article

You just said it!

First of all, engagement optimization is as old as the printing press.

...But the difference now is thay is that news targets algorithms, not eyeballs and reputation.

brucethemoose2··on Ask HN: Using Mistral as encoder for SDXL-like net
You mean as a prompt generator? There are already finetunes for this, though most people use smaller models.

If you are talking about transformers llms that output pixels, that is an area of active research... I hope you have a few million dollars lying around, though.

brucethemoose2··on Stories removed from the Hacker News Front Page, updated in real time
LLMs are like crypto, where scams and scam-adjacents are everywhere.

I am the biggest local ML advocate you will find. My 3090 is either running Yi 34B queries or other experiments all day, my job is with local LLMS... But I am totally OK with heavy handed AI-related moderation. I dont want the sea of AI grifters to have a single second on the HN front page.

brucethemoose2··on Khronos Releases AV1 Decode in Vulkan Video with SDK Support for H.264/H.265
Yeah. What even is the of keeping things split?

RADV even has big commercial customers now (Valve).

brucethemoose2··on Watch a 7k-LB Rivian R1T Destroy a Guardrail in Eyebrow-Raising Test
Yeah, this is what I meant.

Slap taxes above a certain weight, and you will see manufacturers drop the weight to right below the threshold faster than you can blink.

brucethemoose2··on Watch a 7k-LB Rivian R1T Destroy a Guardrail in Eyebrow-Raising Test
I dunno. If you look at modern supercars, they are loaded with luxury stuff, and their mass is noticeably higher than supercars of old when that shouldn't be the case.

I suspect the "race edition" stripped version of many cars are still hauling around the fundamental design features they need to accommodate luxury features.

brucethemoose2··on Watch a 7k-LB Rivian R1T Destroy a Guardrail in Eyebrow-Raising Test
Even a junker can lock its wheels under hard braking, right?

AFAIK the limiting factors are tire grip, ABS control and of course mass.

brucethemoose2··on Watch a 7k-LB Rivian R1T Destroy a Guardrail in Eyebrow-Raising Test
Or maybe cars for commuting shouldn't be 7,500 lb?
brucethemoose2··on Watch a 7k-LB Rivian R1T Destroy a Guardrail in Eyebrow-Raising Test
My beef isn't with EVs, but ballooning car mass in general.

Modern cars are just so huge, and have so much junk in them. I get a large fraction of the mass is for safety, but (for the same money) I'd much rather have a tighter, lighter, stripped down car than a clunky infotainment system, seat coolers, fragile wood trim, motorized ac vents and handles. I'll even sacrifice some sound deadening and rear AC.

And EVs, of course, just make this issue worse. Their "fuel" is such a large fraction of their mass that they more-or-less suffer from the tyranny of the rocket equation, where they need more battery to haul more stuff around, which in turn requires more battery, and now the chassis and suspension needs to be reinforced to handle the weight, and now that needs more battery and so on.

brucethemoose2··on EU chip goal 'unrealistic' says ASML CEO
Random aside, but I love this news site. It's oldschool in the best of ways.
brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
Missed this, thanks.

Everything is moving so fast!

brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
Yeah this is what I use as well. The arbitary quantization is just great, as is everything else.

Its not easy to install though, and big dGPU only.

brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
Analyzing huge documents and info dumps.

Co writing a long story with the model so it references past plot points.

The story writing in particular is just something you can't possibly do well with RAG. You'd be surprised how well LLMS can "understand" a mega context and grasp events, implications and themes from them.

brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
(afaik) it does not support batching, and some other server focused features.

I was talking more about a high throughput server, which is not appropriate for ollama or llama.cpp in general.

brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
Yeah its by far the least trouble. Pretty much any other backend, even a "pytorch free" backend like MLC, is a utter nightmare to install, and that's if it uses a standardized quantization.

However, the llama.cpp server is... very buggy. The OpenAI endpoint doesn't work. It hangs and crashes constantly. I don't see how anyone could use it for batched production as of last november/december.

The reason I don't use llama.cpp personally is no flash attention (yet) and no 8 bit kv cache, so its not too great at long (32K+) contexts. But this is a niche, and being addressed.

brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
Yeah I misinterpreted it open ai as "open source ai"

TBH I do not follow OpenAI much. I like my personal models local, and my workplace likes their models local as well.

brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
They have! And posted them on HN!

Some are pretty good! Check out this little curated nugget: https://llm-tracker.info/

I used to follow one with a UI that resembled HN itself, but now I can't find it in my bookmarks, lol.

brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
The GGML quants are the only quantization we have lol. They were leaked in Q2K/Q4KM/Q5KM, you can grab them right now.

Whats interesting is that Mistral apparently distributed these GGUFs. This is (in my experience) not a good format for production, so I am curious exactly who was wanting to test a model in GGUF.

brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
There aren't really any I know of, because its brand new and everyone just uses vllm :P

No one knows about it! Which is ridiculous because batched requests with loras is mind blowing! Just like many other awesome backends like InternLM's backend, LiteLLM, Outline's VLLM fork, Aphroidte, exllamav2 batching servers and and such. Heck, a lot of trainers don't even publish the loras they merge into base models.

Personally we are waiting on the integration with constrained grammar before swapping to Lorax. Then I am going to add exl2 quantization support myself... I hope.

brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
Lots of interesting information is so fragmented in niche Discords. For instance, KoboldAI on merging and RP models in general, and some other niches. Llama-Index. VoltaML and some others in regards to SD optimization. I could go on and on, and know only a tiny fraction of the useful AI discords.

And yeah, /r/LocalLlama seems to be getting noisier.

TBH I just follow people and discuss stuff on huggingface directly now. Its not great, but at least its not discord.

brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
I have submitted some in the past. Others are submitting them! And I upvote every one I like in /new. But HNers don't really seem interested unless its llama.cpp or Mistral, and I don't want to spam.

I can't say I blame them either, there is a lot of insane crypto-like fraud in the LLM/GenAI space. I watch the space like a hawk... and I couldn't even tell you how to filter it, it's a combination of self-training from experience and just downloading and testing stuff myself.

brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
It's not, its just too much noise to see.

We are getting a lot of great models out of China in particular (Yi, Qwen, InternLM, ChatGLM), and some good continuations like Solar.

Lots of amazing papers on architectures and long context are coming out.

Backends are going crazy. Outlines is shoving constrained generation everywhere, and Lorax is a LLM revelation as far as I'm concerned.

But you won't hear about any of this on Twitter/HN. Pretty much the only thing people tweet about is vllm/llama.cpp and llama/mistral, but there's a lot more out there than that.

brucethemoose2··on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance
Its smart. Definitely keeping it around as my slow/low context local llm.

Seems like a great candiate to merge with other 70Bs as well. There aren't a lot of really great 70B training continuations, like CodeLlama or sequelbox's continuations.

← PreviousPage 5 of 34Next →