eachro

I tried reading this post and it just went way over my head. Anyone have any good resources on background material to even start?

incentive problem

What use cases would you say it is good enough for?

That was surprisingly fun. Nice job!

why is this? I would have thought that more memory would help up to some point (wherever the entire model can fit in RAM) and then no gains after that. I'd also have thought that things like accuracy wouldnt be affected.

Is this not something you can do with chatgpt+?

hows the experience of writing webgpu code? Last I checked some tutorials out, you still had to write the code inside a string without syntax highlighting. Has this changed?

Can someone explain why the numpy code ends up being so slow? Is the author actually doing things in a vectorized manner or just looping over np arrays instead of lists? It's hard to tell from the info given.

I've heard it's grandkids that really tip the scale.

What can a hobbyist do with an FPGA? Ive mostly only heard about fpga's being used at HFT firms somehow.

I believe a precondition for Sam and Greg returning to OpenAI is that the board gets restructured (decelerationists culled). That is probably good for MSFT.

Was Madry at OpenAI for a sabbatical? I thought he was a professor at MIT.

This seems like a terrible decision by OpenAI. How does this benefit them?

I've become partial to interview questions where the interviewee just has to build something like a rock paper scissors game or command line to do list app. Simple prompt, easily extendable as well. It's jarring how many people with 5+ years of experience completely fail on this kind of interview.

Any experienced engineer should have no trouble with it. There's no hiding here. - candidate just needs to deliver something working, something relatively clean, and be reasonably pleasant to pair with. No leetcode grinding necessary, though I have found that those who did well on this problem also generally got high scores from my colleagues who do ask LC questions.

How do you control for hallucinations?

This really brings me back to my days in college, when this was the exact sort of stuff that ML classes focused on. You could have an entire course on various interpretations and extensions of kernel based methods. I wonder how much these insights are worth anymore in the age of LLMs, deep neural networks. I havent kept up too much with the NTK literature but it seems like the theoretical understanding of kernel based methods, gaussian processes does not confer you any advantage in being a better modern ML (specifically working on LLMs) engineer, where the skill set is more heavily geared towards systems engineering and/or devops for babysitting all your experiments.

It's probably fine to have some solutions that operate on different time scales than the other proposed solutions that others are trying.

Yishan Wong's startup Terra formation(https://www.terraformation.com/) is trying to tackle climate change with this more or less as the underlying thesis.

Not particularly relevant but the name "ROCm" is kind of terrible. Hard to pronounce, doesnt look good (the caps and then lower case is quite jarring). Minor details but I feel like these things do have a bit of downstream impact.

As far as I understand, getting robots to do reasonable things when faced with out of distribution experiences is super hard. How do they manage this?

If they don't actually help then the attention weight for that connection would tend to 0, right? Then it becomes a problem of overfitting which we have a large arsenal to combat.

Isn't the natural solution to just use attention layers instead of graph convolution layers then? Then there the attention mechanism will end up learning the useful graph structure via the attention weights?

I'm not sure this is true. For instance, consider reading textbooks for linear algebra and functional analysis out of order. You might still grok the functional analysis if you read it first but you'd be better served by reading the linear algebra one first.

So what is the consensus view on how to do parallelism in python if you just have something that is embarassingly parallel with no communication between processes necessary?

My feed is entirely cooking tiktoks, pottery and restaurant recs for the town I live in. I am fairly ruthless about indicating when I am (like/bookmark) and am not interested (block and hit "not interested" button) in a piece of content.

This is really cool! Thanks for sharing :)

I really enjoy small elegant code demonstrations like this that really allow you to get your hands dirty to try to understand a concept. Another example is Sasha Rush's gpu puzzles, tensor puzzles -https://github.com/srush/GPU-Puzzles -https://github.com/srush/Tensor-Puzzles

Which firm was this?

Leetcode is actually quite good for this. You can find problems to do by topic and the process of doing the easy -> medium problems will get you pretty far in the way of the standard topics (greedy, dynamic programming, divide/conquery, graph search, fundamental data structures and so on).

So practically, how does having this understanding affect the way one thinks about certain probability distributions?