HNHacker News
TopNewBestAskShowJobs

thewataccount

1,110 karma · joined September 1, 2022

submissionscomments
thewataccount··on Linear algebra for programmers
I feel like I saw one once but lost it -

Is there a githubrepo/tutorial for how linear algebra is used for a very small model just to demonstrate how that allows it to "learn"?

I've got the calc, I just don't understand what the matrix multiplication "does"

thewataccount··on Linear algebra for programmers
Can anyone suggest a something that teaches Linear Algebra with a practical applications, especially for software engineering?

I can sorta kinda get the theory, but every demonstration involves moving an arrow around which is.... not something I need to do frequently. So I'm not sure how I actually apply linear algebra to solve actual problems.

I'm a software developer and I know it's useful I just don't get where to use it - and I'm struggling to actually understand the different operations, purpose of the dot product, etc. I have a decent base for basic stats and calc, both of which I can "conceptually apply" near daily for understanding how things work.

3Blue1Brown is helpful, but I just kinda go "yeah I guess that looks right" without knowing what to do with it.

EDIT: Thank you!

thewataccount··on Fine-tuning GPT-3.5-turbo for natural language to SQL
That's for the non-finetuned GPT3.5turbo model.

Finetuned is 1.2cents/1k in and 1.6cents/1k out. So it'll likely be closer to 2cents depending on what you're doing.

I'm not saying it's not useful, at 2c per query you have to be more "purposeful" as they could certainly add up depending on how you use it compared to 0.2c.

thewataccount··on Long Live the 'GPU Poor' – Open-Source AI Grants
Totally understand! I'm sure you'd get swarmed with requests.

Are you interested in non-LLMs as well? Stable diffusion for example?

thewataccount··on Long Live the 'GPU Poor' – Open-Source AI Grants
Hello, I think this is a great project!

If an individual is looking to contribute to the field with different training data ideas - would they need to first establish themselves and get your attention, or would there be a way to submit a proposal?

For myself compute is rather expensive, I could likely afford a few test runs for proofs of concept but beyond that it would be difficult. I've got a single 4090 so I can't run llama70b faster then 1it/s.

thewataccount··on Linux overtakes Mac as Steam's second-most used OS, thanks to the Steam Deck
> fool kernel level anti-cheat

That's the problem though - if the compatibility layer can fool the anti-cheat then so can actual cheats.

Both cheats and compatibility layers do the same thing - modify behaviour.

Generally anticheat will take a "blanket" approach and ban or crash by default and whitelist specific types of behaviour.

What we ultimately would need would be a kernel level interface/process that the game talks to if it identifies it's running under linux. Which is totally possible to do, linux users specifically would hate it, and it would be a lot of work. Without this you're stuck in userland.

Honestly the only realistic hope is the steamdeck becomes a lucrative enough platform for developers to target. Valve has already made sure the popular anti-cheat software supports linux. Valve's stance has been to provide cheat mitigation through other things such as overwatch/trust factor in CSGO (which works well from my experience). VAC itself is actually very primative.

thewataccount··on Ask HN: Why did Python win?
I'll argue that python is very alive in the system management/scripting space.

Because of the library support (especially for data) it's also popular as api servers. It's not as popular for entire applications but is still very active.

EDIT: Django is actually about equal or more popular then nodejs based on google trends. Not sure that means anything as idk how many people google "nodejs" when working on it.

thewataccount··on Linux overtakes Mac as Steam's second-most used OS, thanks to the Steam Deck
> My understanding is there is no technical reason these anti-cheats cannot work via Proton, but the anti-cheat makers will ban if they figure out you're not on Windows out of some weird fear of people having root access to the system. I could be wrong, however.

So the big anti-cheat platforms VAC, EAC, BattleEye, etc. all support linux, however the developer has to enable it - last I checked Battlefield2042 could run on linux just fine (as BF1 and BF5 did) but they didn't enable the anticheat on linux.

I'm not sure how possible kernel level anti-cheat is via wine, the entire point of that is to be on the ring 0 layer where the best cheat software lives. Drivers generally get permissions to read/modify any memory anywhere using hooks that are difficult/impossible to detect from userland.

The reason you'll get banned on linux if they don't instantly nope-out on launch and don't want to support linux is because what anti-cheat software does is primarily two things: See what is hooked into the game, and see if all the memory is in an expected state. Obviously the compatibility tools don't match windows 1:1 and will look _bizzare_ to software comparing it to what you'd expect on windows. So the state doesn't match which it assumes means something is modifying it = cheat software.

tl;dr - idk if ring0 anticheat could ever work via wine, a lot of games use anti-cheat software compatible with linux but just don't enable it, if anti-cheat was somehow didn't instantly freak out on launch then it'll see a bizarre and heavily modified system compare to a real windows install, with unexpected behaviour.

EDIT: From my understanding DKVX for example presents as the direct3d/directx library which of course has very different behaviour and is very clearly modified/not actual direct3d/directx

thewataccount··on Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper
I'm not 100% sure they're talking about this specifically but logit control/manipulation is often used to conform to a specific schema.

https://github.com/guidance-ai/guidance

I'm going to butcher this explanation - after you've generated your selection of logits but before you sample from them, you check which ones conform to your schema. If you want the only two options to be "true" or "false", then you take any of the logits that would provide invalid answers and lower their probabilities manually.

Another example is structures like JSON can be validated so when your sample is "{'name':'Carl'" you lower the probability of "{" since that would invalidate the json. In fact the only valid ones you'd likely have left would be ",", " ", and "}"

thewataccount··on AlloyDB AI: Generative AI applications with PostgreSQL
I think what you're trying to say is that just because someone - especially [large company] - tries to improve/integrate popular open projects doesn't mean it's always EEE.

Which I doubt EEE is purposeful the majority of the time initially, even if has the potential to become that later. In the case of google I think this would be a case of "how do we add value to our product to sell" followed by "this feature costs us too much resources to maintain, let's cut it and focus on [new feature]"

That being said Postgres is not a good example because AFAIK they're both not a commercial entity (or any registered entity at all?) and they're true FOSS. At the moment they have no commercialization strategy (sponsorships maybe?) and no ability to extinguish anything given their open license and lack of any proprietary services.

thewataccount··on Code Llama, a state-of-the-art large language model for coding
I think they're suggesting simply doing +2

+1 is not a good idea since ~half of all numbers are effectively non-prime simply by being even numbers.

You can double the speed by using +2 without using any fancy tricks, just changing a single character.

thewataccount··on Code Llama, a state-of-the-art large language model for coding
Refact has worked for me. Hopefully exllama will support CodeLlama.
thewataccount··on Code Llama, a state-of-the-art large language model for coding
Any plans to support IntelliJ?
thewataccount··on Code Llama, a state-of-the-art large language model for coding
Is automation not what every engineer strives for when possible? Especially software developers.

From my experience with github copilot and GPT4 - developers are NOT going anywhere anytime soon. You'll certainly be faster though.

thewataccount··on Code Llama, a state-of-the-art large language model for coding
Unfortunately what I tested was StarCoder 4bit. We really need exllama which should make even 30b viable from what I can tell.

Because codellama is llama based it may just work possibly?

thewataccount··on Code Llama, a state-of-the-art large language model for coding
For in-editor like copilot you can try this locally - https://github.com/smallcloudai/refact

This works well for me except the 15B+ don't run fast enough on a 4090 - hopefully exllama supports non-llama models, or maybe it'll support CodeLLaMa already I'm not sure.

For general chat testing/usage this works pretty well with lots of options - https://github.com/oobabooga/text-generation-webui/

thewataccount··on 5 years ago Valve released Proton
> Let's assign blame to Nvidia, though.

I'm not going to excuse Nvidia - their driver is buggy and they're still missing DLSS3. Many of the problems I had were nvidia specific, especially anything touching wayland.

However my audio issues, or a bug where if you click the selection of applications when you alt+tab would lock gnome up and you'd have to kill the application from the terminal - those were not nvidia specific AFAIK.

I've also heard of non-smooth experiences from AMD hardware.

I don't think it's fair to recommend linux to anyone who would be scared off by the terminal.

I'm not at all saying it can't be relatively smooth sailing though, especially without a rolling os.

thewataccount··on SeamlessM4T, a Multimodal AI Model for Speech and Text Translation
281M and 235M param models too.

https://github.com/facebookresearch/seamless_communication/b...

I don't really know how the metrics they list compare to whisper, I'm very curious if these are fast enough for realtime speech2text? I think whisper technically could but it was difficult to do or something like that?

thewataccount··on 5 years ago Valve released Proton
I've used linux as my desktop for 2+ years.

The games I play like CSGO work very well*, you have a steamdeck so you know how game support especially with anti cheat is.

However, I've had many bugs. Nvidia on linux is painful. Just linux things - I get a bar at the top of the game when starting sometimes and have to change my resolution back and forth to fix it, I've had to restart pipewire to get my audio to reappear, I've had to replace pulse with pipewire (mid game).

Linux is not smooth, ESPECIALLY with nvidia. If the games you play are well supported, use AMD graphics, and doesn't tinker with their system at all, on a very stable OS - maybe you could call it stable?

Also if you want smooth - for the love of god don't use a rolling distro. If you check the arch wiki you'll see various nvidia/steam/wine/proton issues occur every few weeks. Many completely break playing games for several days unless you downgrade packages.

tl;dr - I would not recommend linux as a desktop to anyone who doesn't mind having their nose in their terminal desperately trying to figure out why you have no audio while your friends grow tired waiting for you.

The steamdeck specifically is very well managed by valve and I'm incredibly impressed that they made it work so well.

thewataccount··on Textual: Rapid Application Development framework for Python
Long running processes only need to load their imports, configuration, database connections, etc. once, and don't incur overhead for each request. On top of this, things like imports get to be cached, and there's less need for a database connection pooler to broker connections.

Particularly async runtimes have become more popular - where you can have a single process handling concurrent requests and awaiting/yielding in between their api/db calls. Instead of having many processes that don't share resources that might be waiting 10ms/100ms for api/db calls.

For applications that make large numbers of api/db calls - especially slower ones, may prefer async runtimes over individual processes per request.

I'm not at all saying CGI is a bad option, just that there are real benefits to reusing, pooling and caching resources.

thewataccount··on Ask HN: LLM Leaderboard for Code Generation?
You're looking for "HumanEval" tests. Not saying this is the best way to test it, but it's the only standard test I know of that code models are compared with and are commonly benchmarked for

The current best models you'd want to try that I'm aware of is WizardCoder(15B), Starcoder(15B), and replit's code model(3B). Replit's instruct model is interesting because of it's competitive performance while only being a 3B model so it's the easiest/fastest to use.

Perhaps interestingly none of these are based on LLama

https://github.com/abacaj/code-eval - This is a large mostly up to date list of benchmarks

https://huggingface.co/WizardLM/WizardCoder-15B-V1.0 - has a chart with a mostly up to date comparison

EDIT: License-wise I think you might be able to commercially use Replit's model and Starcoder, I don't think you're allowed to use WizardCoder outside of academic work.

thewataccount··on D1 receptors may be a more important target than D2 for antipsychotic drugs
I have some bad news for you about Tylenol/Paracetamol....

The science is "We did many double blind studies on a large population and observed these effects to a degree statistically significant compared to placebo"

We can reproduce the effects reliably, and the side effects are similarly predictable. We've also researched the pharmacokinetics to see what it's doing, but we don't necessarily understand how those changes cause it's effects.

Just because we don't know exactly how it works, doesn't mean we didn't observe what the effects are. When the good effects out way the bad effects then it's a possible treatment.

I'm not saying all studies are of quality, that there aren't unknown effects we haven't observed yet, that there aren't problems etc. What I'm saying is we have at least attempted to measure the effects and determine them to a statistically significant degree.

tl;dr - If we've measured effects to a statistically significant degree - that is not a cargo cult. How many people understand the electrical wiring in their house yet use a light switch daily?

thewataccount··on Rhino Linux is a rolling release Ubuntu-based distribution
> It's really nice when every compiler/language is on the latest version by default.

I've seen breaking changes in my dependencies from 3.7 -> 3.8 -> 3.9 -> 3.10, both at work and at home.

I've also had constant issues with both GCC and CUDA being too new.

The reverse is true too of course - I use a rolling release specifically because of a new CUDA version that no stable distro has (or will have for many months)

thewataccount··on StableCode
Its worth noting that from what I can tell - A model well trained in most languages would be able to learn the niche ones much more easily.

The vocab size of llama2 is 32,000. I guess I personally don't think that there's enough difference in programming languages to actually save any meaningful number of tokens considering the magnitude of the current vocab.

thewataccount··on Supabase Local Dev: migrations, branching, and observability
I checked a little bit ago but I might have missed it so please correct me.

Are there plans to expand self-hosting support? The migrations are a big step forward.

Are you intending to fill the roll of being a framework akin to a "super django" type of deal? Again the migrations help a ton, but I've been hesitant to use Supabase for random projects because I don't want to rely on the platform, and I don't want random people on github who want to try or contribute to require a supabase account.

I'd love to use it more as a modular ORM for miscellaneous projects instead of the current "hosted platform", currently none of the tutorials (or github projects) seem to explain this route at all.

I think you actually do work for this purpose, and I think the docs mostly cover the bare minimum for self hosting, and I understand your business kinda relies on the hosted platform, but I'd love to see further tutorials and thorough explinations of all the features - currently some like the AI features aren't really explained if they work in the self-hosted or not, or if you have to do anything special for that.

EDIT: Also I do appreciate your business being opensource, and contributing to postgres so much! Sorry for the rambling, and I apologize if I'm blind and missed some obvious docs.

thewataccount··on Testing Intel’s Arc A770 GPU for Deep Learning
Check out the hugging face leader board and try one of the finetunes.

I can't run llama2 70B - but I remember llama1-Guanaco30B was a major leap over the base llama30B model for me.

thewataccount··on PyPI Requires 2FA for New User Registrations
Any chance of signed builds returning? It's bizarre to me that we would move _away_ from signed builds.

2FA means we can trust the person that logged in - but we still don't trust that PyPI is being honest (no offense).

thewataccount··on StableCode
I think simply having the vocab more code friendly (e.g. codex) would make the biggest difference, whitespace is the biggest one (afaik every space is a token), but consider how many languages continue `for(int i=0;`, `) {\n`, `} else {`, 'import ', etc.

My understanding is that a model properly trained on multiple languages will beat an expert based system. I feel like programming languages overlap, and interop with each other enough that I wouldn't want to specialize it in just one language.

thewataccount··on StableCode
I get decent performance with my 4090, enough that LLMs with exllama at 30B quantitized are very usable. But we're severely VRAM limited, especially on lower end hardware which rarely sees > 10GB of VRAM.

I don't know how much slower it could be and still be useful though. The big thing is we need more VRAM, 30B is context length limited with only 24GB of vram, I've only barely made it above 3.2k tokens before running out.

I hope you're right, that it becomes common for systems to have either dedicated TPU type stuff similar to smartphones, and that they absolutely load the crap with VRAM (which I don't think is even that expensive?)

Models will also get smaller but I'm skeptical we'll get GPT4 performance with any useful context length under 24GB VRAM any time soon.

thewataccount··on Project IDX: An experiment to improve full-stack, multiplatform app development
I'm not excited about developing "in the cloud".

Privacy, speed, lock-in, etc.

Especially with google's track record.

← PreviousPage 2 of 12Next →