SuchAnonMuchWow

Twist: the real capcha is detecting if the user first press on "click to start"

actual source of the article: https://gofetch.fail/

> I feel like this follows naturally from having only ternary values, multiplication doesn't really bring much to the table here. It's a bit surprising that it's performing so well on existing hardware, usually multiplication hardware sees more optimization, especially for GPGPU hardware.

No, unless I'm mistaken it's a huge impact: it means the matrix product is separable: basically, it's a O(n²) algorithm, and not O(n3): add together all the c_j = sum(a_i_j), d_i = sum(b_i_j), and the final results are all the combinations of cj+di. And even then, half that is unnecessary because the d_i can all be pre-computed when before inference since they are weights.

But I skimmed over the paper, and didn't found the part where it was explained how they replace the product by additions: from what I understand, they remplace multiplications by bi by selecting +ai, 0, or -ai. So the final matrix multiplication can be implemented by only additions, but only because the weights are 1,0,-1 they avoid multiplications altogether. This is really different from what the GP said (remplacing a0*b0+... by a0+b0+...).

Wait til you learn what "web 3" meant before cryptoscams started to also use this term.

The issue with their approach is that the whole LLM must fit in the chips to run at all: you need hundreds of cards to run a 7B LLM.

This approach is very good if you want to spend several millions building a large inference server to achieve the lowest latency possible. But it doesn't make sense for a lone customer buying a single card, since you wouldn't really be able to run anything on it.

Simone Giertz is usually years in advanced compared to Tesla. See Truckla vs. Cybertuck: years of advance, and a much more attractive car.

Depending on your library of valid transition, it could be really easy to find a path. That's the issue: there are libraries of paths where finding a path is really hard, but libraries where finding a path is easy.

Say in another way, we know only the worst case complexity, not the complexity of a test case taken randomly.

Some latices-based cryptosystems have such properties: there are theorems showing that if there is an algorithm good at solving random cases, than you can derive from it an algorithm good at solving any cases, including the worst ones. So for those problems, the worst case complexity is the same than the "average/random case" complexity.

Honestly, given the crappy direction that windows is taking, I'm ready to abandon it the minute apple can run games seamlessly. And I doubt I'm the only one.

Skip connection increase the live range of one intermediate result across the whole part of the network skiped: the tensor at the beginning of a skip connection must be stored in memory for longer while unrelated computation happen: it increase the pressure on the memory hierarchy (either the L2, or scratchpad memory).

This is especially true for example for inference for vision transformers, where it decrease the batch size you can use before hitting the L2 capacity wall.

Just in case you're open to something new: Tree Style Tabs is my goto extension to managing tabs on firefox

You manage your tabs vertically in a sidebar, organized as a tree. Every new tab is opened as a leaf from the tab that you came from, grouping your tabs by branches following your navigation on the web. This make managing groups of tabs by topics really natural.

This exploit the fact that the game use a PRNG instead of a true RNG, and can work regardless of whether the PRNG is a CSPRNG or not.

It doesn't seems to be related to the controller: the same phison controller was in some model that failed and some models that passed.

(see later in the thread: https://twitter.com/xenadu02/status/1496770658750980103)

Except that view point fell even earlier when they refused to release their models after GPT-2.

They already use their dominance in one platform (android) to achieve dominance in another (chrome), the exact same way Microsoft have been found guilty of years ago.

If the US govt wanted to sue google for it, they would have years ago.

I'm not sure what you mean: ASCII text encoding is full of controls characters, most of them being only historical and not used by anything, including terminals.

From around 2005, they provided free hosting with a custom subdomain with apache, php, mysqL, and ~10GB of disc space. At that time it was miles above other free hosting offers, so a ton of blogs/personal/small websites used them.

That's true. Integers of this form a+b*sqrt(5) are called quadratic integers (https://en.wikipedia.org/wiki/Quadratic_integer), and define a ring for which you can define addition and multiplication rules:

(a+b*sqrt(5)) + (c+d*sqrt(5)) = (a+c) + (b+d)*sqrt(5)

(a+b*sqrt(5)) * (c+d*sqrt(5)) = (ac+5bd) + (ad+bc)*sqrt(5)

Note that seeing the operations this way reduce a bit the number of operations compared to the matrix representation you provided: a multiplication using the formula above require 5 integer multiplications instead of 8 with the matrix [[a, 5b], [b, a]].

But this is only a constant factor improvement: the algorithm complexity is still log(n).

And the same result can be achieved by using the special structure of the matrix: for your representation, the matrix are all toeplitz matrices, and multiplying those kind of matrices require less operations than full generic matrix multiplication. For the representation in the original article, If I remember correctly, the matrices are triangular, and again you can multiply them more efficiently using this specificity.

The main issue with floating point number is that it doesn't act like a ring (https://en.wikipedia.org/wiki/Ring_(mathematics)) because of rounding errors. For example the equality x*(y*z) = (x*y)*z doesn't hold because each multiplication introduce small rounding errors.

And this kind of equations are required for the exponentiation-by-squaring method (consider for example how x**5 is computed: x**5 = (((x*x)*x)*x)*x) using a naive exponentiation approach, but also x**5 = x*(x*x)*(x*x) = x*(x*x)**2 using the exponentiation-by-squaring method).

Integer operations and matrix operations are both rings, so the exponentiation-by-squaring approach works. For floating points, it doesn't give an exact result, but could still be used in practice to get an approximate result.

The pow function doesn't use this approach however, because both arguments of the pow function are floating points numbers, and can thus compute things like pow(5, 0.5) = sqrt(5), or even 534.2**12.7. The exponentiation-by-squaring approach works only when the exponent is an integer, so we need other solutions.

How exactly the pow function (or other math function like exp, log, cos, ...) are computed is a vast research subject in itself, but if you are interested, the Handbook of Floating-Point Arithmetic (https://perso.ens-lyon.fr/jean-michel.muller/Handbook.html) is both a good introduction to the domain (and the problems associated with the finite precision of floating point number), and provides a lot of techniques to work around or minimize the rounding errors that happens when manipulating floating point numbers.

The comparison with the closed form is weird to me: since it uses sqrt(5), I suppose the author intended to use floating point numbers to compute it. But when doing so:

- You get a constant time algorithm, since the power function is constant time on most mathematical librarie. Without entering into details on how the pow function is actually computed, on real number you can compute it with: pow(x,y) = exp(y * log(x)). It is not visible on the performance graph probably because the matrix-based algo and the closed formula are dwarfed by the linear time algo.

- You gen an incorrect result very quickly: not even considering the fact that sqrt(5) can not be represented exactly by a floating point number, the result will be out of the range of floating point numbers with n proportional to the number of bits your floating point number holds, so probably between n=50 and n=100.

> I really fail to see what people see even in Nvidia/Cuda

Cloud providers are choosing nvidia today as the gpgpu platform on the cloud, driven by neural network needs. So lots of software will be written with the platform in mind, aka. for nvidia arch / in cuda.

This will add friction preventing users to easily change platform (see how long it took for people to switch their workloads to arm for example). It will drives more cloud providers to use nvidia, more software to be written for cuda, and the moat will build itself.

No, functionally complete is mussing the looping functionality to be Turing complete.

Turing complete is often misused to say functionally complete, either because people mistake the two or because it makes for a more appealing blog post / article:

- mov is in fact not turing complete: it needs a jmp instruction (https://harrisonwl.github.io/assets/courses/malware/spring20...)

- homomorphic encryption systems are functionally complete but not Turing complete (since looping leaks the number of operations done, break the encryption)

I use twitch quite extensively and I almost never see ads. Only maybe 2-3 per year when the cat temporarily catch the mouse, and then the mouse free itself again. No fancy setup either: just a ublock origin on firefox.

While its true that atomic can solve the issue of atomic operations (increment, compare and swap, ...) that volatile doesn't try to solve, it is also required to solve the issue that volatile tried but failed to solve correctly: you have no ordering guaranteed between volatile and non-volatile memory accesses (see problem no.5 of the article).

In that way, atomic complete volatile instead of being orthogonal o it, because it provides the ordering semantic missing in volatile. And it doesn't replace it completely, because as you said, atomic accesses can still be optimized away.

So in most use-cases of volatile, you actually want to declare your variables atomic+volatile along with the correct memory_order on your atomic operations.

Note that "bad trip" is the french word for a (drugs related) bad trip, like "weekend" is the french word for weekend. So that explain a bit the name of the movie.

What the OP is missing in its "today" analysis is that that cloud platform are choosing nvidia right now since its the most mature compute platform, and so the software for using GPU on the cloud will be written more and more in cuda / using nvidia libs: it will become a defacto standard, and nvidia will entrench themselves that way.

crushed by technical debt

It probably is: lead paint is much more durable and resistant, which is why it was everywhere in the 50s

Considering the previous tweet said "fuck you", yes it is an attack.

In the link provided, the CUDA example only show the compute kernel itself and not the boilerplate required to run it. On the other hand, your OpenCL example only show the boilerplate.

This is the OpenCL kernel from the same repo, for a more fair comparison: https://github.com/rsnemmen/OpenCL-examples/blob/master/mand...

This is much more readable. OpenCL-C the language is fine: it's how you deploy the program on the cards that is complicated with opencl.

About the area, keep in mind that not every part of a chip is composed of dense logic:

Typically, sram (cache memories for example) doesn't scale as much as core logic, and interconnect (connections between the cores, DDR, etc...) almost don't scale at all.

This is highly dependent on the design of the SoC, but the core logic is rarely the largest of the three, so you will not see a 35% reduction of the total area of the chip: only of the areas the most densely packed with logic. What you could have for example is a 35% more powerful neural network accelerator in the same area as the previous generation. But you will not have 35% bigger cache.