Is there a githubrepo/tutorial for how linear algebra is used for a very small model just to demonstrate how that allows it to "learn"?
I've got the calc, I just don't understand what the matrix multiplication "does"
1,110 karma · joined September 1, 2022
Is there a githubrepo/tutorial for how linear algebra is used for a very small model just to demonstrate how that allows it to "learn"?
I've got the calc, I just don't understand what the matrix multiplication "does"
I can sorta kinda get the theory, but every demonstration involves moving an arrow around which is.... not something I need to do frequently. So I'm not sure how I actually apply linear algebra to solve actual problems.
I'm a software developer and I know it's useful I just don't get where to use it - and I'm struggling to actually understand the different operations, purpose of the dot product, etc. I have a decent base for basic stats and calc, both of which I can "conceptually apply" near daily for understanding how things work.
3Blue1Brown is helpful, but I just kinda go "yeah I guess that looks right" without knowing what to do with it.
EDIT: Thank you!
Finetuned is 1.2cents/1k in and 1.6cents/1k out. So it'll likely be closer to 2cents depending on what you're doing.
I'm not saying it's not useful, at 2c per query you have to be more "purposeful" as they could certainly add up depending on how you use it compared to 0.2c.
Are you interested in non-LLMs as well? Stable diffusion for example?
If an individual is looking to contribute to the field with different training data ideas - would they need to first establish themselves and get your attention, or would there be a way to submit a proposal?
For myself compute is rather expensive, I could likely afford a few test runs for proofs of concept but beyond that it would be difficult. I've got a single 4090 so I can't run llama70b faster then 1it/s.
That's the problem though - if the compatibility layer can fool the anti-cheat then so can actual cheats.
Both cheats and compatibility layers do the same thing - modify behaviour.
Generally anticheat will take a "blanket" approach and ban or crash by default and whitelist specific types of behaviour.
What we ultimately would need would be a kernel level interface/process that the game talks to if it identifies it's running under linux. Which is totally possible to do, linux users specifically would hate it, and it would be a lot of work. Without this you're stuck in userland.
Honestly the only realistic hope is the steamdeck becomes a lucrative enough platform for developers to target. Valve has already made sure the popular anti-cheat software supports linux. Valve's stance has been to provide cheat mitigation through other things such as overwatch/trust factor in CSGO (which works well from my experience). VAC itself is actually very primative.
Because of the library support (especially for data) it's also popular as api servers. It's not as popular for entire applications but is still very active.
EDIT: Django is actually about equal or more popular then nodejs based on google trends. Not sure that means anything as idk how many people google "nodejs" when working on it.
So the big anti-cheat platforms VAC, EAC, BattleEye, etc. all support linux, however the developer has to enable it - last I checked Battlefield2042 could run on linux just fine (as BF1 and BF5 did) but they didn't enable the anticheat on linux.
I'm not sure how possible kernel level anti-cheat is via wine, the entire point of that is to be on the ring 0 layer where the best cheat software lives. Drivers generally get permissions to read/modify any memory anywhere using hooks that are difficult/impossible to detect from userland.
The reason you'll get banned on linux if they don't instantly nope-out on launch and don't want to support linux is because what anti-cheat software does is primarily two things: See what is hooked into the game, and see if all the memory is in an expected state. Obviously the compatibility tools don't match windows 1:1 and will look _bizzare_ to software comparing it to what you'd expect on windows. So the state doesn't match which it assumes means something is modifying it = cheat software.
tl;dr - idk if ring0 anticheat could ever work via wine, a lot of games use anti-cheat software compatible with linux but just don't enable it, if anti-cheat was somehow didn't instantly freak out on launch then it'll see a bizarre and heavily modified system compare to a real windows install, with unexpected behaviour.
EDIT: From my understanding DKVX for example presents as the direct3d/directx library which of course has very different behaviour and is very clearly modified/not actual direct3d/directx
https://github.com/guidance-ai/guidance
I'm going to butcher this explanation - after you've generated your selection of logits but before you sample from them, you check which ones conform to your schema. If you want the only two options to be "true" or "false", then you take any of the logits that would provide invalid answers and lower their probabilities manually.
Another example is structures like JSON can be validated so when your sample is "{'name':'Carl'" you lower the probability of "{" since that would invalidate the json. In fact the only valid ones you'd likely have left would be ",", " ", and "}"
Which I doubt EEE is purposeful the majority of the time initially, even if has the potential to become that later. In the case of google I think this would be a case of "how do we add value to our product to sell" followed by "this feature costs us too much resources to maintain, let's cut it and focus on [new feature]"
That being said Postgres is not a good example because AFAIK they're both not a commercial entity (or any registered entity at all?) and they're true FOSS. At the moment they have no commercialization strategy (sponsorships maybe?) and no ability to extinguish anything given their open license and lack of any proprietary services.
+1 is not a good idea since ~half of all numbers are effectively non-prime simply by being even numbers.
You can double the speed by using +2 without using any fancy tricks, just changing a single character.
From my experience with github copilot and GPT4 - developers are NOT going anywhere anytime soon. You'll certainly be faster though.
Because codellama is llama based it may just work possibly?
This works well for me except the 15B+ don't run fast enough on a 4090 - hopefully exllama supports non-llama models, or maybe it'll support CodeLLaMa already I'm not sure.
For general chat testing/usage this works pretty well with lots of options - https://github.com/oobabooga/text-generation-webui/
I'm not going to excuse Nvidia - their driver is buggy and they're still missing DLSS3. Many of the problems I had were nvidia specific, especially anything touching wayland.
However my audio issues, or a bug where if you click the selection of applications when you alt+tab would lock gnome up and you'd have to kill the application from the terminal - those were not nvidia specific AFAIK.
I've also heard of non-smooth experiences from AMD hardware.
I don't think it's fair to recommend linux to anyone who would be scared off by the terminal.
I'm not at all saying it can't be relatively smooth sailing though, especially without a rolling os.
https://github.com/facebookresearch/seamless_communication/b...
I don't really know how the metrics they list compare to whisper, I'm very curious if these are fast enough for realtime speech2text? I think whisper technically could but it was difficult to do or something like that?
The games I play like CSGO work very well*, you have a steamdeck so you know how game support especially with anti cheat is.
However, I've had many bugs. Nvidia on linux is painful. Just linux things - I get a bar at the top of the game when starting sometimes and have to change my resolution back and forth to fix it, I've had to restart pipewire to get my audio to reappear, I've had to replace pulse with pipewire (mid game).
Linux is not smooth, ESPECIALLY with nvidia. If the games you play are well supported, use AMD graphics, and doesn't tinker with their system at all, on a very stable OS - maybe you could call it stable?
Also if you want smooth - for the love of god don't use a rolling distro. If you check the arch wiki you'll see various nvidia/steam/wine/proton issues occur every few weeks. Many completely break playing games for several days unless you downgrade packages.
tl;dr - I would not recommend linux as a desktop to anyone who doesn't mind having their nose in their terminal desperately trying to figure out why you have no audio while your friends grow tired waiting for you.
The steamdeck specifically is very well managed by valve and I'm incredibly impressed that they made it work so well.
Particularly async runtimes have become more popular - where you can have a single process handling concurrent requests and awaiting/yielding in between their api/db calls. Instead of having many processes that don't share resources that might be waiting 10ms/100ms for api/db calls.
For applications that make large numbers of api/db calls - especially slower ones, may prefer async runtimes over individual processes per request.
I'm not at all saying CGI is a bad option, just that there are real benefits to reusing, pooling and caching resources.
The current best models you'd want to try that I'm aware of is WizardCoder(15B), Starcoder(15B), and replit's code model(3B). Replit's instruct model is interesting because of it's competitive performance while only being a 3B model so it's the easiest/fastest to use.
Perhaps interestingly none of these are based on LLama
https://github.com/abacaj/code-eval - This is a large mostly up to date list of benchmarks
https://huggingface.co/WizardLM/WizardCoder-15B-V1.0 - has a chart with a mostly up to date comparison
EDIT: License-wise I think you might be able to commercially use Replit's model and Starcoder, I don't think you're allowed to use WizardCoder outside of academic work.
The science is "We did many double blind studies on a large population and observed these effects to a degree statistically significant compared to placebo"
We can reproduce the effects reliably, and the side effects are similarly predictable. We've also researched the pharmacokinetics to see what it's doing, but we don't necessarily understand how those changes cause it's effects.
Just because we don't know exactly how it works, doesn't mean we didn't observe what the effects are. When the good effects out way the bad effects then it's a possible treatment.
I'm not saying all studies are of quality, that there aren't unknown effects we haven't observed yet, that there aren't problems etc. What I'm saying is we have at least attempted to measure the effects and determine them to a statistically significant degree.
tl;dr - If we've measured effects to a statistically significant degree - that is not a cargo cult. How many people understand the electrical wiring in their house yet use a light switch daily?
I've seen breaking changes in my dependencies from 3.7 -> 3.8 -> 3.9 -> 3.10, both at work and at home.
I've also had constant issues with both GCC and CUDA being too new.
The reverse is true too of course - I use a rolling release specifically because of a new CUDA version that no stable distro has (or will have for many months)
The vocab size of llama2 is 32,000. I guess I personally don't think that there's enough difference in programming languages to actually save any meaningful number of tokens considering the magnitude of the current vocab.
Are there plans to expand self-hosting support? The migrations are a big step forward.
Are you intending to fill the roll of being a framework akin to a "super django" type of deal? Again the migrations help a ton, but I've been hesitant to use Supabase for random projects because I don't want to rely on the platform, and I don't want random people on github who want to try or contribute to require a supabase account.
I'd love to use it more as a modular ORM for miscellaneous projects instead of the current "hosted platform", currently none of the tutorials (or github projects) seem to explain this route at all.
I think you actually do work for this purpose, and I think the docs mostly cover the bare minimum for self hosting, and I understand your business kinda relies on the hosted platform, but I'd love to see further tutorials and thorough explinations of all the features - currently some like the AI features aren't really explained if they work in the self-hosted or not, or if you have to do anything special for that.
EDIT: Also I do appreciate your business being opensource, and contributing to postgres so much! Sorry for the rambling, and I apologize if I'm blind and missed some obvious docs.
I can't run llama2 70B - but I remember llama1-Guanaco30B was a major leap over the base llama30B model for me.
2FA means we can trust the person that logged in - but we still don't trust that PyPI is being honest (no offense).
My understanding is that a model properly trained on multiple languages will beat an expert based system. I feel like programming languages overlap, and interop with each other enough that I wouldn't want to specialize it in just one language.
I don't know how much slower it could be and still be useful though. The big thing is we need more VRAM, 30B is context length limited with only 24GB of vram, I've only barely made it above 3.2k tokens before running out.
I hope you're right, that it becomes common for systems to have either dedicated TPU type stuff similar to smartphones, and that they absolutely load the crap with VRAM (which I don't think is even that expensive?)
Models will also get smaller but I'm skeptical we'll get GPT4 performance with any useful context length under 24GB VRAM any time soon.
Privacy, speed, lock-in, etc.
Especially with google's track record.