stu2b50

Yes, but with more criteria than "cheap knockoff". There's a lot of companies these days that satisfy the niche of "printers that print well and quickly out of the box without tinkering". Furthermore, many of them are priced below Prusa's offerings.

"Cheap knockoff" implies a significant degradation in quality, but the reality is that they are of equal quality, and lower price - the fact that Prusa is open source and has a good reputation as a brand can make up ground, but in the end many consumers mainly care about meeting a certain standard of performance, and then sorting by ascending price.

Prusa went from niche to fairly mainstream by being the printer that "just works". Mainstream is way more money than the enthusiast, hobby segment of the 3d printing market. So they would feel the burn from having those sales lost.

From a pragmatic point of view, I think there's some functional issues with what Prusa is trying to do, though. For one, it implies that the "clones" will always only get there by cloning your own technology, which we know isn't the case from every other industry where this has happened.

Bambu printers already have significant technological advantages over Prusa printers. The people there [at Bambu and other companies] aren't dumb - once bootstrapped by examining open source printers, they can and will learn to build good printers in their own right.

So when Prusa starts to become less open, which they inevitably must do if the premise is that there are companies that don't particularly care about foreign IP rights and licenses, then they just enter the rat race with companies that are quite frankly just better at making things for less money.

Maybe it is just financially unviable to be the open source standard bearer, but in that case it's likely just financially unviable for the company to exist.

This doesn't really seem like a real issue. By eschewing domain names, you've already killed any chance of a mainstream audience for the site. That's fine, but it's also not particularly complicated for the users to support an alternative protocol where you don't need TLS if it's already so niche.

That nets you revenue, not profit.

Negative externalities require external interference in market economies to produce the socially optimal outcome. Climate change and pollution are clear examples of negative externalities.

As it happens, the farming of meat actually has severe negative externalities in this case. If anything, it's farmed meat that should be restricted by the government in that model.

Claude from Anthropic already has an API and its likely that Bard will have an API as well. It's just the most basic way to monetize your model.

That's not what dropout is, dropout is just a method of regularization (you remove a random, and small, subset of the neurons per training iteration, in order to reduce overfitting), and pretty much all LLM transformer blocks have dropout.

They have the LoRA delta weights on huggingface, which is linked on the github. Since it's the just the deltas, they're substantially smaller (~8mb), and you'll need to supply the original 7b LLaMA yourself.

It's not an issue of copyright as terms of service. The images you create out of a pirate photoshop, you do own the copyright, but adobe can also go after you for the unrelated matter of illegally bypassing their DRM.

I mean the derivative of a constant is 0. So if all of the original weights are considered constants, then computing their gradients is trivial, since they’re just zero.

In terms of PCA, PCA is also quite expensive computationally. Additionally, you'd probably have to do SVD instead.

Since the weights are derived from gradient descent, yeah we don't really know what the distributions would be.

A random projection empirically works quite well for very high dimensions, and is of course very cheap computationally.

You’re assuming a lot more intercompany coordination than would exist. Even though it’s research by Microsoft labs, the researchers themselves are to a large extent autonomous and also narrow experts in their fields.

This process involves low rank approximations -> Lora is a namey sounding term that uses characters from low and rank -> call it LoRA in the paper. That’s all there was to it. Probably didn’t even know the other lora existed.

I suppose that is true. You can even train the prompt with gradient descent. But in practice, it ends up being fairly different.

> But this is smaller than the alternative of a model which contains the large matrix of original weights, and an equally large matrix of alterations.

It's actually larger. If you just have two equally large matrices of the same dimension, one original, and one of "altercations"... then you can just add them together.

> Why is fine-tuning done with separate alterations, rather than by mutating the original weights?

Then you'd have to compute the gradients for the whole network, which is very expensive when the model has 7b, 65b, 165b parameters. The intent is to make that cheaper by only computing gradients for a low rank representation of the change in the weight matrix from training.

Random projects work well in high dimensional spaces, they’re cheap, easy, and require no understanding of the initial space. Part of the point of Lora is efficiency, after all!

It cheapens the cost of fine tuning, it doesn’t make the model itself smaller at inference time.

It’s actually completely different. What you linked is about zero shot learning by adjusting the prompt, vs Lora which is about actually fine tuning the weights of the model.

Per the original paper, empirically it’s been found that neural network weights often have low intrinsic rank. It follows, then, that the change in the weights as you train also have low intrinsic rank, which means that you should be able represent them with a lower rank matrix.

I suspect it’s not that similar. The intuition behind LoRA is more true the higher the rank of the weights of the model. Even the smallest LLMs have considerably higher rank weights than Stable Diffusion. They are large, after all.

Empirically, while the ability for LLMs to zero shot learn is impressive, it’s significantly worse than fine tuning. An obvious example is LLaMA itself, from which it’s quite hard to get useful instructional behavior out, and requires a significant amount of prompt engineering, and is still brittle at that.

Fine tuning it in just 52k examples (alpaca) makes a night and day difference in usability for instruction following.

I don’t think text is sufficiently unconstrained. It is trivial to get GPT to output “Hi, how are you?” but of course it’s impossible to determine if that common turn of phrase is human or machine generated.

There is too much a correlation between the position of tokens and correct meaning. In contrast, in an image, it’s very possible to have pixels shuffle around and not change the look. There, it is more likely that we can determine that the output is statistically more likely to be from a particular model.

The issue with latex is that I don't personally use it every day, and I suspect many people are in that camp. When you just use it ocassionally, all that syntax just falls out of your brain, leading to a frustrating loop where you're constantly relearning latex.

In comparison, I use vim keybindings every day. As a result, despite how esoteric it may be, it's well ingrained into my muscle memory.

Well, it’s more that the weights are the neurons. There’s not actually like neuron objects defined or anything, neural networks are just a bunch of matrix operations. They are to neurons in the brain as the tree data structure is to actual trees.

> Copilot LLMs are not trained on your tenant data or your prompts. Within your tenant, our time-tested permissioning model ensures that data won’t leak across user groups. And on an individual level, Copilot presents only data you can access using the same technology that we’ve been using for years to secure customer data.

2 of the 3 demo videos is literally a student asking it arithmetic problems

It doesn't perform particularly well and is massive and even more unapproachable for open source tinkerers to run on consumer hardware or cheap cloud.

Llama performs better on benchmarks while a fraction of the size.

OP's claim was that Stripe Capital's loan underwriter was SVB, hence why they weren't extending loans to SVB affected customers.

Stripe itself was, and probably still is, using Wells Fargo for US corporate accounting, per https://qr.ae/pvERsZ

I don't think the implication is that the junior engineer should be carefully calculating their coworker's hourly rates and estimating the dollar value of what your slack DM to them will cost the company. Just that you do a good enough job that your work brings in more money than your salary costs.

I think they bank with Celtic Bank. I think this because it says on the Stripe Capital product page on the bottom: https://stripe.com/capital/platforms

> Loans are issued by Celtic Bank, a Utah-Chartered Industrial Bank Member FDIC. All loans subject to credit approval.

VRAM is the thing that Apple Silicon is going to have in excess compared to anything even close in price. MacBook Airs can have 14-15GB of VRAM if necessary.