jacobn

So a lot of Apple Intelligence is still vaporware then?

From [1] "Additional Apple Intelligence features will roll out later this year and in the months following", i.e. later in 2024 and then in 2025, and the not-available-at-launch features appear to be the stronger ones, e.g. ChatGPT integration?

Any word on improving the dictate-to-text keyboard feature? I could only find "In the Notes and Phone apps, users can also record, transcribe, and summarize audio", but that's different.

[1] https://www.apple.com/newsroom/2024/09/apple-debuts-iphone-1...

The fact that they have touch id on some of the ipad side power buttons, but not on the iphone, is so annoying. Face id is both great and terrible, and complementing it with touch id to cover the >5% of cases where face id fails would be such an improvement.

The only number I could find was 22/27 hours of video playback for 16/16 Pro, you can compare that to older models on https://www.apple.com/iphone/compare/ that might give some indication?

Nice work! Making AI accessible directly in the browser (even if you have to download quite the payload for the model parameters) is a real accessibility game changer.

Shameless plug: if you prefer API-based background removal we offer a super low cost, high quality option on https://pixian.ai

I was under the impression that you could more or less do something like that with the existing LLMs?

(May work poorly of course, and the sample I think I saw a year ago may well be cherry picked)

I agree it sounds absolutely absurd - first time I heard about it I couldn’t believe it.

The Wikipedia page has more on the rationale, good or bad:

https://en.wikipedia.org/wiki/Civil_forfeiture_in_the_United...

Any confiscated cash needs to go into state level or even federal coffers, not to the agency or local cops that did the confiscating. That removes the immediate incentive for abuse, while retaining the tool for what it was intended for: actual criminals.

(I don’t have the sources but read years ago that states that let the cops keep the cash have ridiculous levels of abusive confiscations, whereas states that have the money go to the state coffers basically don’t)

(Most civil asset forfeiture is done by local cops stopping people on the highway and similar)

Is it known why/how those animals have reduced / negligible senescence? Like do they have some other variant of the AP-1 gene(s), or some such?

At the AI engineer fair, or some other event?

Tangentially related: I needed to share some binary data from Java to python. Didn’t want to pull in protobuf (for probably mostly bad reasons), so I wrote a writer implementation in Java, then asked gpt4o to write the python reader. Worked like a charm - it produced a great starting point then refactored it to my liking.

All that to say: with chatbots being able to implement “the other side”, it really made it a lot easier to just whip up a custom schema and not really worry too much.

(Helped that it was a one off in my case, unclear how well this would scale to real world production usage)

A very big reason for the “go big or go home” mentality comes from very simple company valuation metrics:

If you have $10M ARR and modest growth you’re probably going to get a ~4x P/E valuation.

If you have $100M ARR and modest growth you can go public and get a ~20x P/E valuation.

So you’re not only 10x more valuable, you’re 50x more valuable.

Basically you get a massive multiplier boost on top of the increased scale.

(Rough & approximate, YMMV, etc)

> Goes to show just how much is in the training data.

And in the scale (num_layers, embed_dim, num_heads) of the model of course ;)

Forgot one: the positional encoding also changed, llama3 uses RoPE, gpt2 uses a learned embedding.

The training code is presumably quite a bit more complex than what they've open sourced, but part of the beauty of the GPT-based LLMs is their structural simplicity.

Now, that simplicity can be deceiving - there are a lot of conceptual interconnectedness within these models. They've been put together "just so" if you will.

If you look at the source code to nanoGPT and compare it to Llama3, the most remarkable thing (when you look past the superficial name changes) is just how similar they are.

If I recall correctly the primary differences are:

  - The MLP: Llama3 uses SwiGLU vs the more "traditional" x = x + proj(gelu(expand(x))) in GPT2
  - The token encoders, which is arguably external to the model
  - Attention: Llama3 uses Grouped Query Attention, vs full Multi-Head Attention in GPT2
  - Normalization: Llama3 uses RMSNorm, vs LayerNorm for GPT2

They were published more than five years apart. On the one hand progress has been breathtaking, truly astounding. On the other hand, it's almost exactly the same model.

Goes to show just how much is in the training data.

In transformers the attention matrix is N*N, so there are a lot of values to go over. Typically makes it memory bandwidth bound, not compute bound.

Can we all agree to just call it X-Twitter? Clear & concise.

Unless of course: “Public perceptions of higher education have declined rapidly, and I expect the problems to get worse.” turns out to be the case

(Which it probably won’t)

Which probably corresponds to a loss of foot traffic to restaurants & shops in the cities' downtowns. This is very visible in SF.

I've been mulling over this weird idea that we should all be using an Edge (Web) Framework that manages the layouts + static site content, then pulls in either JSON or pre-rendered HTML templates from the more dynamic db-driven backends (which can then be in any language, and mapped to any path, and can be a majestic monolith or many microservices, whichever).

It would free things up so much - the backend team can focus on JSON/HTML templates, the designers can work directly with the website without necessarily going through dev (for better or worse ;), marketing can add all their little trackers, easy to outsource parts, and you could build up an entire cross-language ecosystem of plugins that manage anything from user accounts, billing, wikis, blogs, knowledge base, admin CRUD systems, ... yet it's all on the same site from the end user's perspective.

I think there are multiple enterprises that have (also re-)invented this architecture, but I haven't seen any serious open source offering that focuses on this type of use.

Ethics are a lot more nuanced and change a lot faster than laws.

Heck, a large fraction of ethics seem to be so fickle that they’re subject to potential revision by every generation.

In fact, I’d argue that those revisions are a significant portion of how one generation distinguishes itself from their parents.

Yet strangely every generation feels like they have arrived at a set of “universal laws” in their ethics.

The fundamental operation done by the transformer, softmax(Q.K^T).V, is essentially a KV-store lookup.

The Query is dotted with the Key, then you take the softmax to pick mostly one winning Key (the Key closest to the Query basically), and then use the corresponding Value.

That is really, really close to a KV lookup, except it's a little soft (i.e. can hit multiple Keys), and it can be optimized using gradient descent style methods to find the suitable QKV mappings.

“we attribute the performance of Yi models primarily to its data quality resulting from our data-engineering efforts”

Data work is rarely sexy, but (almost) always useful.

Did they release the corpus?

This is a “sounds good, but devil is in the details” type situation, and the details typically don’t work out all that well because the initiative effectively assumes good intent from the ecosystem, when we have seen time and time again that there are enough hostile actors with enough creativity to spoil it.

Reviewing vs creating are very different mindsets. There’s a red team, blue team flavor to it.

AI levels up the less skilled, but makes even the more skilled miss embarrassing mistakes.

SEPA via Stripe takes on average 5 days, max 14 according to their docs - instantaneous would be a game changer?!

https://stripe.com/docs/payments/sepa-debit

Establishing democracy is indeed also a fragile process, but one that has numerous successful implementations (~140 democracies in the world?).

Social democracy has numerous successful implementations (eg Scandinavia).

But the more “hardcore” collectivist -isms have from what I can recall basically zero successes, despite numerous attempts.

The problem with collectivism is less with collectivism as platonic ideal (which has a lot of romantic appeal), and more with the inevitable concentration of power it requires and thereby enables.

The reason there has been no successful implementation of it is that the process of enacting it is inherently fragile. And some bastard is going to exploit that fragility.

Then you're screwed and we end up with the bountiful historical examples that people love to cite & then others refute as "that's not real xyz-ism".

You're absolutely correct that they're not "real" xyz-ism. And that should make you very worried about supporting something that has the same or similar end goals as those initiatives once had.

Interesting point that abundant cheap-but-bad products can effectively choke off the second hand market. It complements the boots theory nicely [1].

[1] https://en.wikipedia.org/wiki/Boots_theory

It seems like a lot of the problems arise from the strings being attached to the top of the pin

Why not attach them to the bottom and have the setting machine underneath?

More exactly how does a proxy add an online DDL capability to the presumably remote database?

Or is a database proxy something completely different than eg a web proxy?