pulse7

Educate me: how to create a program in Goose which reads a JSON structure, prints it out and finishes. How is memory for JSON allocated on stack? It can be from 1K to 1M - depending on the file...

Prompt processing on Macs is VERY slow...

I hope we will soon have an open-source project for training such small LLMs where one can just pick the architecture (like Qwen / DeepSeek / etc.), parameter count, dataset, ... and then let it run on a local/rented GPUs...

"code translation in hardware" microcode is called "native"

So it's mostly the "Optionality". Like USB. And yet USB is everywhere...

It there anything similar for RTX 3090 and RTX 4090?

Can you please tell which Gemma 4 variant managed to correctly reason through your private benchmarks? Was is Gemma 4 31B?

What quantizations and context lengths did you use for Gemma 4 and Qwen 3.8 27B?

I am asking because I can't even load Gemma 4 31B on my GPU with any reasonable quantization (even with small context), while I can run Qwen 3.8 27B with large context and good quantization...

"--context-shift, --no-context-shift ... whether to use context shift on infinite text generation (default: disabled)"

From: https://github.com/ggml-org/llama.cpp/blob/master/tools/serv...

It will come... all big hardware players (Intel, AMD, Broadcom) and dozens of startups (Tenstorrent, etc.) are working on it...

Hasn't Docker always been just a thin layer of duct tape over existing solutions?

You cannot wear out an SSD through AI inference alone. LLM weights are only read from the SSD, and read operations do not contribute to SSD wear. SSD wear is primarily caused by write and erase operations.

Those are routers - not general purpose computers.

On the absolute limit of ethics? This is simply not ethical anymore...

The same here! Merged several repos into monorepo are retained full history!

Most probably not optimized yet for this model...

With electron-beam lithography you can build transistors with gate lengths down to 1 to 3 nanometers.

exactly: high demand!

NVidia Spark is much slower (low memory bandwidth)!

Can you please share you llama.cpp server parameters to turn on modern LLM sampling stack?

Docs [1] say that the top_n_sigma is already in the default sampler list: "(default: penalties;dry;top_n_sigma;top_k;typ_p;top_p;min_p;xtc;temperature)"

[1] https://github.com/ggml-org/llama.cpp/blob/master/tools/serv...

This https://en.wikipedia.org/wiki/Elon_Musk%27s_Tesla_Roadster ?

I'm sorry your dad didn't respect your IT work...

So it's similar to "Andy and Bill's Law" [1]: "What Intel giveth, Microsoft taketh away".

If Windows would stay the same (and not grow) it would be much faster on newer CPUs...

[1] https://en.wikipedia.org/wiki/Andy_and_Bill%27s_law

It looks like the president - which was a businessman - will make a huge damage to American IT businesses. And IT stocks dominate the S&P 500, comprising roughly 1/3 of the index's total market capitalization... Good luck America!

Source?

Maybe they can stack LLM parameters in 200 layers like 3D NAND flash and make the chip very small ...

IBM was founded in 1911 and it survived many things...

...and there would be dozen equally capable open-weight models which could be run locally at almost no cost... poor AI investors in this case..

Nvidia is the ultimate beneficiary of the money invested (due to expensive GPUs). If Nvidia loses these good customers, it will have less revenue. So it prefers to slowly buy it's customers with this money...

<joke> GGUF when? </joke>

I'm very happy that "AGI office workers" will use Microsoft products - so I don't have to do it anymore... But: they will not pay a dime for the licenses...