DougBTX

S3 doesn't format the size for human display, the object's Size is returned in bytes:

https://docs.aws.amazon.com/AmazonS3/latest/API/API_Object.h...

Yeah, US-centric, I wouldn’t have thought of that. My search results page for Plan B has a musician, a film, climate change advocacy... but the ads are all for pills.

> The actual process of token generation works precisely the same

I’d be wary of generalising it like that, it is like saying that all programs run on the same set of CPU instructions. NNs are function approximators, where the code is expressed in model weights rather than text, but that doesn’t make all functions the same.

How do generics change this? Can a type bound on a generic work like a static interface implementation specification, and recover the cost of wrapping the struct?

To me it says that they _do_ train global models with customer data, but they are trying to ensure no data leakage (which will be hard, but maybe not impossible, if they are training with it).

The caveats are for “local” models, where you would want the model to be able to answer questions about discussions in the workspace.

It makes me wonder how they handle “private” chats, can they leak across a workspace?

Presumably they are trying to train a generic language model which has very low recall for facts in the training data, then using RAG across the chats that the logged on user can see to provide local content.

No idea of this is even vaguely in the right direction, but this comes to mind: Unsupervised speech-to-speech translation from monolingual data

https://research.google/blog/unsupervised-speech-to-speech-t...

There is literally a whole bunch of people very focused on narrow definitions. It seems like cognitive dissonance trying to square the idea that a dog which can fetch a ball is intelligent but a computer which can write poetry isn’t.

Such a strict interpretation would also exclude spaces.

Not really compilers, but the underlying GPU libraries.

Here’s a good jumping off point: https://pytorch.org/docs/stable/generated/torch.use_determin...

Same, location identified by the architecture of the buildings in the background and the car's numberplate... of a car with no numberplate driving through a wood.

Yes, probably. At temperature zero the model will be completely deterministic, so a particular prompt will always produce the same result (ignoring for a second that some fairly common optimisations introduce data races in the GPU).

On the other hand, does it really matter? With a slight tweak to the prompt, ChatGPT generates some serviceable code:

    > Run a function to produce a random number between 1 and 10. What is the number?

    import random

    # Generate a random number between 1 and 10
    random_number = random.randint(1, 10)
    random_number

    The random number generated between 1 and 10 is 9.

No, an algorithm may be written in many ways, each of which would encode to a different Gödel number.

> To encode an entire formula, which is a sequence of symbols, Gödel used the following system. Given a sequence (x_1, x_2, x_3, ..., x_n) of positive integers, the Gödel encoding of the sequence is the product of the first n primes raised to their corresponding values in the sequence

https://en.wikipedia.org/wiki/Gödel_numbering

There are also FAISS binary indexes[0], so it'd be great to compare binary index vs binary index. Otherwise it seems a little misleading to say it is a FAISS vs not FAISS comparison, since really it would be a binary index vs not binary index comparison. I'm not too familiar with binary indexes, so if there's a significant difference between the types of binary index then it'd be great to explain what that is too.

[0] https://github.com/facebookresearch/faiss/wiki/Binary-indexe...

Absolutely, have been for decades, see expert systems: https://en.m.wikipedia.org/wiki/Expert_system (I take the strong position that code is data, so an if statement in code is equivalent to an if-then rule stored in a database: it is just a different way of storing the data. However others would take a different position on that.)

The issue is that encoding human knowledge in if statements is hard to scale, so it doesn’t seem to be on the path to general artificial intelligence, but fundamentally it is still automation of cognition.

Agreed, we shouldn’t use “invariant” as a shorthand for “not expected to change”.

We can still talk about “implicit assumptions” and “explicit assumptions”, so the article still works after that replacement. The headline isn’t as punchy though.

The projection logic has an orientation parameter (which you'd probably always want, so that you can pick initial rotation), so the rotation animation itself is just updating the param on a timer.

Basically line 182 plus the calls to redraw:

    gamma -= 0.003;

To do it with CSS, at a minimum it would require adding support for rendering to two different canvas elements.

Probably just because it is harder to implement, the CSS would be an optimisation of a specific case on top of the general code. Classic cost (in time and overall complexity) vs performance tradeoff.

Would this be a reasonable explanation?

> MLPs are universal function approximators, but these models are big enough that it is better to train many small functions rather than a single unified function. MoE is a mechanism to force different parts of the model to learn distinct functions.

> largest integer in the original list

So if the original list was [10, 1] and negative numbers were allowed, then [10, 2, -1] would be allowed.

This might interest you: https://transformer-circuits.pub/2023/monosemantic-features

An embedding is essentially a set of neuron activations, the number of combinations of activations is much bigger than the number of neurons themselves. An embedding could store "A+B" as well as being able to store "A" and "B", but it is a little complex to understand what particular activations mean.

That’s an easy one: the company exists to provide The Services, revenue from commercialising the content supports the company.

Biggest fine from last year was apparently $1.2 billion.

The mechanism is described in the FAQ here: https://gofetch.fail

> The GoFetch attack is based on a CPU feature called data memory-dependent prefetcher (DMP), which is present in the latest Apple processors. We reverse-engineered DMPs on Apple m-series CPUs and found that the DMP activates (and attempts to dereference) data loaded from memory that "looks like" a pointer. This explicitly violates a requirement of the constant-time programming paradigm, which forbids mixing data and memory access patterns.

> To exploit the DMP, we craft chosen inputs to cryptographic operations, in a way where pointer-like values only appear if we have correctly guessed some bits of the secret key. We verify these guesses by monitoring whether the DMP performs a dereference through cache-timing analysis. Once we make a correct guess, we proceed to guess the next batch of key bits. Using this approach, we show end-to-end key extraction attacks on popular constant-time implementations of classical (OpenSSL Diffie-Hellman Key Exchange, Go RSA decryption) and post-quantum cryptography (CRYSTALS-Kyber and CRYSTALS-Dilithium).

It sounds like the OpenSSL code is still constant time (it doesn't expect pointers in the intermediate values, to OpenSSL it is just data, not something it will ever dereference) but the attacker-controlled "monitoring" code's runtime changes based on whether the DMP ran or not.

If that's right, then the attacker still needs to run their monitoring code on the target, it isn't sufficient to just run OpenSSL etc itself.

Edit: it is more explicit in the paper, they assume that OpenSSL (the victim) is still constant time:

> In this paper we assume a typical microarchitectural attack scenario, where the victim and attacker have two different processes co-located on the same machine.

> For our cryptographic attacks, we assume the attacker runs unprivileged code and is able to interact with the victim via nominal software interfaces, triggering it to perform private key operations. Next, we assume that the victim is constant-time software that does not exhibit any (known) microarchitectural side-channel leakage. Finally, we assume that the attacker and the victim do not share memory, but that the attacker can monitor any microarchitectural side channels available to it, e.g., cache latency.

I’ll admit to not having looked at the details at all, but a possible explanation is that almost all the time is spent on inter process communication overhead, so if that also happens before the timer starts (eg, the data has been transferred, just waiting to be read from a local buffer) then the measured time will be significantly lower.

Another way to look at it is that concatenating files should sum the line count. Concatenating two empty files produces an empty file, so 0 + 0 = 0. If “incomplete lines” are not counted as lines, then the maths still works out. If they counted as lines, it would end up as 1 + 1 = 1.

It depends on where the timing code is. If the timer starts after all the data has already been loaded, the time recorded will be lower (even if the total time for the whole process is higher).

One man’s hallucination is another’s creativity.

Interesting! And "Science" is all the way down in Class 500.

https://en.wikipedia.org/wiki/List_of_Dewey_Decimal_classes

I’ve just tried with ChatGPT on iOS, you can press stop then immediately respond while the AI is still generating the response.

No, either this is fair use, so no license is required, or the current licences must be followed (including attribution which seems like the first thing that gets forgotten).