coder543

In the original announcement of the Siri revamped a couple of years ago, they specifically talked about having the on-device model handle everything it can, and only using the cloud models for the harder or more open ended questions.

Yes, new ones will be uploaded. That doesn’t mean old ones won’t also be downloaded.

It absolutely works the way I said it does, because I have seen it work that way. Just because you accidentally turned off iCloud Photos in your Apple Account settings on that Mac (or some other similar issue) does not mean it does not work this way when properly signed in.

If you want something to try, go to System Settings -> Apple Account -> Photos and see if "Sync this mac" is turned off. It needs to be on. There could be other ways that this feature is disabled, but that is one of them.

Not seeing something work is not evidence that it does not work. You have not seen it work, but that is not proof it does not work.

Seeing it work is evidence that it works. I have seen it work.

Other people have seen it work that way, and their replies are all over this thread. Apple documents that it works this way.

Yes, it will upload photos to iCloud if enabled, but it also downloads them.

As long as you are signed into the Mac with the same iCloud account used on the iPhone, this will download them all. No, you do not need to get them all downloaded to the iPhone ever for any reason for this to work. Period. You need to stop repeating that, because it is wrong. How many people have to say the same thing?

Yes, you will have to go into a hidden folder to access the Originals once they're downloaded if you want to copy them somewhere else, but it's like two clicks.

Even if Tailwind were a shared hosted system like the common bootstrap CDNs of old… CDNs are dirt cheap for a small text file, even if it were loaded billions of times a month.

Some back of the napkin math suggests that it would cost about $300 per billion downloads for the current bootstrap.min.css file (gzip compressed, naturally) at North American network prices on one CDN I’ve used before. Or just $150 per billion globally if you're willing to use fewer PoPs. With browser caching, even split per domain, a billion downloads covers a very large number of users for a very large number of page loads.

In the olden days of 2023, you didn’t just find instruct-tuned models sitting on every shelf.

You could use a base model that has only undergone pretraining and can only generate text continuations based on the input it receives. If you provided the model with several examples of a question followed by an answer, and then provided a new question followed by a blank for the next answer, the model understood from the context that it needed to answer the question. This is the most primitive use of ICL, and a very basic way to achieve limited instruction following behavior.

With this few-shot example, I would call that few-shot ICL. Not zero shot, even though the model weights are locked.

But, I am learning that it is technically called zero shot, and I will accept this, even if I think it is a confusingly named concept.

Sorry I tried to help? If that's the response I get for helping, good luck...

> All I meant was a way to avoid storing images in git, the rest is quite simple.

There is no good way to do that, and no way that I would recommend. Git is the correct solution, if that is where you are storing the markdown. No fancy git tools are required.

Usually you just store the images in the same git repo as the markdown. How you initially host the static site once generated is up to you.

The problem with storing binaries in Git is when they change frequently, since that will quickly bloat the repo. But, images that are part of the website will ~never change over time, so they don't really cause problems.

The Llama 4 models are MoE models, in case you are unaware, since it feels like your comment feels was implying they were dense models.

On the flip side, it also shows how damaging echo chambers can be, where relatively few people even gave the models a chance, just repeating the negativity they heard from other people and downvoting anyone who voiced a different experience.

I think this was exacerbated by the fact that Llama models had previously come in small, dense sizes like 8B that people could run on modest hardware, where even Llama 4 Scout was a large model that a lot of people in the community weren’t prepared to run. Large models seem more socially accepted now than they were when Llama 4 launched.

I can get about 20 tokens per second on the DGX Spark using llama-3.3-70B with no loss in quality compared to the model you were benchmarking:

    llama-server \
        --model      llama-3.3-70b-instruct-ud-q4_k_xl.gguf \
        --model-draft llama-3.2-1b-instruct-ud-q8_k_xl.gguf \
        --ctx-size      80000 \
        --ctx-size-draft 4096 \
        --draft-min 1 \
        --draft-max 8 \
        --draft-p-min 0.65 \
        -ngl 999 \
        --flash-attn on \
        --parallel 1 \
        --no-mmap \
        --jinja \
        --temp 0.0 \
        -fit off

    prompt eval time = 313.70 ms / 40 tokens (7.84 ms per token, 127.51 tokens per second)
    eval time = 46278.35 ms / 913 tokens (50.69 ms per token, 19.73 tokens per second)
    total time = 46592.05 ms / 953 tokens
    draft acceptance rate = 0.87616 (757 accepted / 864 generated)

Prompt processing is literally 3x to 4x higher on GPT-OSS-120B once you are a little bit into your context window, and it is similarly much faster for image generation or any other AI task.

Plus the Nvidia ecosystem, as others have mentioned.

One discussion with benchmarks: https://www.reddit.com/r/LocalLLaMA/comments/1oonomc/comment...

If all you care about is token generation with a tiny context window, then they are very close, but that’s basically the only time. I studied this problem extensively before deciding what to buy, and I wish Strix Halo had been the better option.

You're better off prioritizing the offload of the KV cache and attention layers to the GPU than trying to offload a specific expert or two, but the performance loss I was talking about earlier still means you're not offloading enough for a 96GB GPU to make things how they need to be. You need multiple, or you need a Mac Studio.

If someone buys one of these $8000 GPUs to run GLM-4.7, they're going to be immensely disappointed. This is my point.

The M4 Max has double the memory bandwidth, so it should be faster for decode (token generation).

If you are running a REAP model (eliminating experts), then you are not running GLM-4.7 at that point — you’re running some other model which has poorly defined characteristics. If you are running GLM-4.7, you have to have all of the experts accessible. You don’t get to pick and choose.

If you have enough system RAM, you can offload some layers (not experts) to the GPU and keep the rest in system RAM, but the performance is asymptotically close to CPU-only. If you offload more than a handful of layers, then the GPU is mostly sitting around waiting for work. At which point, are you really running it “on” the RTX Pro 6000?

If you want to use RTX Pro 6000s to run GLM-4.7, then you really need 3 or 4 of them, which is a lot more than $10k.

And I don’t consider running a 1-bit superquant to be a valid thing here either. Much better off running a smaller model at that point. Quantization is often better than a smaller model, but only up to a point which that is beyond.

One RTX Pro 6000 is not going to be able to run GLM-4.7, so it's not really a choice if that is the goal.

The point in this thread is that it would likely be too slow due to prompt processing. (M5 Ultra might fix this with the GPU's new neural accelerators.)

400k, not 256k.