eachro

I've been looking into getting into GPU programming, starting with CS334 (https://developer.nvidia.com/udacity-cs344-intro-parallel-pr...) on Udacity. I'm curious to hear from some of the more seasoned GPU veterans out there, what other resources would be good to take a look at after finishing the videos and assignments?

I've veen a bit out of the loop on this area but would like to get back into it given how much has changed in the LLM landscape in the last 1-2 yrs. What models are small enough to play with on Collab? Or am I going to have to spin up my own gpu box on aws to be able to mess around with these models?

Ah I should have been a bit more clear. I'm interested in how FHE actually works and the steps needed to transform general computation to its FHE equivalent.

I see HN pieces on SIMD optimizations for numerical computing every so often. Are these the sort of optimizations that a hobbyist with say a normal laptop (either a macbook air or consumer grade thinkpad) can actually end up tinkering with as well? Or am I going to have to rent some AWS machine and have it run on some specific architecture to be able to mess around with?

At this point there are quite a lot of companies training these massive LLMs. We're seeing startups with models that are not quite GPT-4 level but close enough to GPT-3.5 pop up on a near daily basis. Moreover, model weights are being released all the time, giving individuals the opportunity to tinker with them and further release improved models back to the masses. We've seen this with the llama/alpaca/alpaca.cpp/alpaca-lora releases not too long ago. So I am not at all worried about this risk of corporate control.

Does anyone know of a good reference to get up to speed on FHE in ML?

If this makes it easier to build new buildings/housing, I'm all for it.

I'd have expected speech recognition software to be good enough that you could have direct speech -> gpt type services. It almost feels like traveling to the past when asking things of Siri when I otherwise get very useful responses from chatgpt.

There is absolutely a reason ...

Is it faster than webgpu? If so, what makes it so?

Knowing matrix derivatives well is one of those skills that were essential in machine learning 10 years ago. Not so much anymore with the dominance of massive neural networks.

Not having an explicit ranking system means that things are ranked by submission time. This too incentivizes certain behavior (frequent posting). Maybe one can get around that with certain guardrails like rate limiting submissions/comments but that's yet another can of worms to open.

What makes it approximately permutation equivariant (vs entirely)? As I understand things, if the order is jumbled, the attention matrix does get its rows and cols permuted in the way you'd expect so I'd have thought they'd be entirely permutation equivariant.

Suppose you're a ML practictioner. Would you still recommend learning WebGPU, over say spending more time on CUDA?

I'm not so sure. Does anyone pay for pytorch, numpy, tensorflow? In the matter of weeks we've seen llama.cpp, alpaca.cpp released to the public. Barrier to entry in this market is quickly going to zero.

Open source is a race to the bottom. Seems like the only obvious winner then is people selling the shovels aka NVIDIA.

What does mmap do exactly? Why was the transition to using it a big improvement in llama.cpp?

I think people who have played sports can attest to how much easier (and less physically demanding it is) to play defense in sports like basketball/soccer when you are able to see where the play is developing.

I thought Raschka was a tenured professor. Did he leave academia?

Coderpad was a one man show for a while I believe.

Is there a big difference between chatgpt and chatgptplus? I use chatgpt for routine things every day (some basic word smithing, looking up how to use libraries, etc) and it is already quite good. What does the 20/mo get me that I don't already get with the free version?

How does LoRA save more than 50% of the memory usage? I see that the weight updates have much lower memory footprint by virtue if being low rank. But you still need the dense weights for the forward pass dont you?

Sorry for the noob question but what can you do with web assembly that you wouldnt otherwise do with other web frameworks? What are people using web assembly for generally?

Does someone know how the llama.cpp was implemented? Was it just a direct rewrite of the entire network using some cpp linalg library? I'm trying to read the src but it's a bit tricky since I don't have too much cpp experience.

How do they pay for their compute?

I generally prefer vim to vscode. But being able to jump to the file where highlighted functions/classes/etc are defined in vscode is very convenient and something I am not able to do in vim. I'm sure there's probably a vim plugin that can allow me to do this too but I just haven't spent the time to look for it.

I'm curious what the true cost of dairy/soy/oat milk would be without gov subsidies.

Depends on your background and what you're most interested in learning.

- if you want to really understand the mathematical foundations, you should take Andrew Ng's course (the full Stanford course not the Coursera one)

- If you want a quick sampler of ML and to know the big ideas without investing too much time, take Andrew Ng's coursera course. I'm not sure what his deeplearning.ai course is like but it seems like that one is in python.

- if you want to learn by getting your hands dirty, either fast.ai's ML course or Dive into Deep Learning is good. Fast.ai's course will get you into the nitty gritty parts of training neural networks.

Who said anything of their desire to return to work?

Havent noticed anything different.

I've found that it's been also pretty good at writing sql queries, and pinpointing where input queries are incorrect. It's probably my highest leverage use of chatgpt at the moment.