HNHacker News
TopNewBestAskShowJobs

ismailmaj

297 karma · joined August 9, 2022

ML Infra

[[username]][[hardy number]] at g mail

submissionscomments
ismailmaj··on Rust SIMD on the GPU
There is something very SIMD-coded in GPU programming which is coalesced stores/loads, if a warp (32 threads) handles contiguous memory, it will create ~4 transactions instead of 32.
ismailmaj··on U.S. economy lost 23,000 jobs in July, a sudden reversal
Given the trend of revisions, I wouldn't be surprised if the real loss is around 6 figures, May 2026 revision went from 172k to 63k.
ismailmaj··on Why Large Language Models Fail at Tabular Prediction
Unsure if it's LLMs that fail at tabular data or its just that tree boosting are spectacular at that task.
ismailmaj··on AI doesn't generate working products, that's still your job
For my projects, I can produce with LLMs a codebase that I'd be proud of, but it requires extensive use of the plan mode and nagging anytime the implementation looks much more complex than the feature at hand requires.

Anytime I tried spec driven development, it produced total garbage, the problem is that the model usually swings too far, if they write unnecessary complexity, a nag about it and you risk it code golfing, this problem across 100 lines of specs and you're guaranteed it will swing too far on some segments you just wanted X slightly more than Y (usually for me it was reliability dropped for better readability since # users = # developers = 1)

Another thing I dropped is steering implementation design too early, start with the goal and nag in the direction you want, it works well for me but might not work well for big complex codebases.

For now I love LLM coding but I'm sure my opinion will change if I have to review code from a developer that does the 1 prompt = 1 PR without looking at the code jutsu.

ismailmaj··on Advancing the price-performance frontier with GPT‑5.6
it's 80% less cost, not 80% in efficiency gains, could be that Luna was overpriced to begin with, we don't have much info on the models themselves.

Assuming the efficiency gains are real, I feel like something has to give, maybe worse quality due to aggressive quantization/kv cache compression?

ismailmaj··on LearnVector – Andrew Ng's AI company building one‑to‑one learning experiences
While the model itself is great at hiding the answer, the AI summary keeps giving it away, it's so annoying and AFAIK it cannot be disabled.
ismailmaj··on Claude Opus 5
I'd pay good money to see OpenAI "oh fuck" war rooms.
ismailmaj··on Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
I think I would be horrified looking at your Claude API bill.
ismailmaj··on GPT-5.6
Apparently plus users do not have access to Sol, so I'm really worried about the ugly Terra Pelican.
ismailmaj··on A software engineering interview question I like: computing the median
I don't know what the day to day would be like but it was a ML research engineer role.
ismailmaj··on A software engineering interview question I like: computing the median
I'm pretty confident I would've been able to come up to the solution in better circumstances, maybe even without hints.

But it was clear in this case that the interviewer just took a question from the company's bank of questions and was alt-tabbed for half the interview, I have felt the energy early and I was also half-checked out.

I'm aware I'm saying this post factum, but I had a very fun first interview with that company and matched well with the first interviewer so my expectations were high, and then I got hit by the big tech style interview when it was an early stage startup.

ismailmaj··on A software engineering interview question I like: computing the median
I got that recently, I really didn't like that question.

For the n-th percentile version, the obvious solution is sorting and it takes 10 seconds to get to that point, 5 minutes of implementation with tests. Good. It's all downhill from here.

Then you get hit with the "it's a data stream" and you realize you have to implement a balanced tree on the spot which I wouldn't describe as fun.

You may or may not be able to implement that. I did not. Blabbered something about Rust having sorted B-Trees and I don't think Python has them -- they do not on the standard library.

Then the interviewer leaned heavily on the "reduce memory usage" and I couldn't come up with a solution (no shit it's Ω(n) and he didn't even tell me to go fetch for a randomized algorithm). I later understood he expected the reservoir sampling solution which is basically keeping a representative group of size K that is a good proxy of the whole stream, it goes like this: keep the K first elements, any elements after that replaces any element of our sample at random.

What I did after 10 minutes of weird silence is to assume the data stream follows a normal distribution and computing the P-percentiles by computing the running mean and standard deviation.

I felt frustrated at the end of the interview because it really felt like a big gotcha of either you know the reservoir sampling "leetcode trivia" or you don't.

ismailmaj··on Tell HN: Who wants to be hired" posts outpace "Who's hiring" 2 to 1
High interest rates and instability makes it a bad environment to invest, it raises the bar on speculative/growth investment, and tech is mainly that.

Mix that with heavy AI bills, there isn't a lot of budget left for hiring.

ismailmaj··on Leaking YouTube creators' private videos
From talking to someone that worked at YouTube for 15 years, they still had a lot of core Python code in 2016 that was legacy from the OG company/team and that code needed to be transitioned to follow the Google way of doing things in C++/Go.

I don't think it was distinct enough from the Google culture like Android was at the start of the acquisition but it seems they had leeway to do their own thing.

ismailmaj··on Leaking YouTube creators' private videos
I'll pick small company, thank you.
ismailmaj··on Fable 5 Is Back
I'm not getting the Opus 4.8 switch for coding, supposedly given how fast I reached the usage limit, which is kind of nice.
ismailmaj··on Zig's new bitCast semantics and LLVM back end improvements
It's great for defining fancy floats used in machine learning

e.g. https://github.com/zml/zml/blob/33ced8fa078b3c7c8c709bd526ae...

ismailmaj··on OpenAI unveils its first custom chip, built by Broadcom
they're still kings for training, though I've heard Anthropic is training now on JAX+TPU setup, so might not be a monopoly in that segment.
ismailmaj··on Challenging the Narrative of European Decline
I can talk about France specifically, the working age population feels squeezed currently because a big portion of their wages goes to pay pensions, let's be specific:

for a 100k paid by your employer, 46k goes to you after all taxes, 45k goes to social contribution, of those 20k specifically to paying current pensioners[1].

It doesn't look so bad, unless you take into account the inversion of the demographic pyramid, the workers will get progressively more squeezed as older folks enter retirement, and are unlikely to get themselves good retirement given that there isn't enough kids to support them when they will be old.

And so, the rational move is to save more to fund your retirement while already getting pressured by relatively low wages and low saving potential due to the taxes, and France doesn't really have a growth lever to pull to get us out of this situation as we're already at a very high debt-to-GDP ratio.

So that's why some workers are unhappy with what they call "fat pensions", what also doesn't help in France is that the country is very centralized around Paris and rent is now another channel where money flows from workers to (often) pensioners.

As 2027 elections approaches, it seems we have the options of being even more fiscally irresponsible (the extremes), and the center seems to lean for austerity measures (Edouard Philippe) and none of these are making me optimistic about the future.

[1]: https://code.travail.gouv.fr/outils/simulateur-embauche

ismailmaj··on Challenging the Narrative of European Decline
> The risk of being cut off from strategically important technologies, once minimal, is now very real.

Good timing given the Fable 5 story.

ismailmaj··on Statement on US government directive to suspend access to Fable 5 and Mythos 5
My main read on this is that European governments will more aggressively invest in regional AI labs.

For Nvidia chips you could've deluded yourself that the US is just anti-china, that position is harder to argue for right now.

ismailmaj··on Ask HN: Are most corporate SWE jobs performative?
In my experience a lot of companies try really hard to be data oriented and try to find objective metrics for impact, sometimes it’s good, often it’s bad. Like LOC count, PR count, time in meetings or time spent at the office.

Enough of this and people will learn to play the game over doing the right thing.

ismailmaj··on I am giving up on VM Gaming
I had a much better experience gaming on MacOS than I ever did using Linux + X (it was first dual boot with Windows, then double GPU + VM, then proton).

But it's mainly because I do not target anymore games with stringent anti-cheats or with high setup requirements (though the M5 is quite powerful right now), and a lot of games are ported natively to macOS but not Linux (most recent to date is Age of Empire 2 Definitive Edition 1 week ago).

ismailmaj··on Gemma 4 12B: A unified, encoder-free multimodal model
Gemini is a huge team while Gemma is relatively small. They can totally do this at a loss with no ulterior motive.

They remind me a bit of HuggingFace, create something great then make money … maybe.

ismailmaj··on Real-time LLM Inference on Standard GPUs: 3k tokens/s per request
I expected a 4090, maybe 2. I did not expect 8xH200 for a 2B model.
ismailmaj··on Claude Opus 4.8
I just asked the model details about the incoming spaceX IPO and it responded with “There’s no confirmed SpaceX IPO. Elon Musk has said for years that SpaceX itself won’t go public”. It took me two push backs and specifically asking for web search.

I feel like I won’t like this model just like I didn’t like 4.7, push backs a lot and avoids thinking or search as much as possible.

ismailmaj··on OpenAI Is Preparing to File for an IPO Soon
3 things, they can, there is a precedent for that with Google v. Oracle for Java, and they have something!

AMD engineered something called HIP which is CUDA API compatible libraries that targets AMD's hardware, it's the closest thing we have for drop-in replacement to Nvidia's software moat.

It works for simple stuff but loses terribly for frontier kernels (like Flash Attention 3), novel approaches (e.g. Mamba) or networking (e.g. NCCL), also they are rough on the edges, so what you gain from GPU costs is lost in engineering cost.

My previous company tried to compete in this GPU game while putting effort to have a good software stack (Rivos), drop in replacement and cheaper with decent software.

But that vision was rough, any new player had to implement the bad APIs due to backward compatibility concerns, following specs wasn't sufficient as a lot of the AI stack was depending on observable effects (Hyrum's Law), and Nvidia simply just had a long head start, the company is now dead (acquired by Meta) and AFAIK there isn't another player.

Best case scenario AMD puts more effort into their software stack but I just think they do not have enough internal talent to compete.

Training will continue to be an Nvidia's thing and that's where most of the money sits, unless suddenly the AI research scene pivots to using JAX but I do not see it coming any time soon, if anything, I've seen internal efforts at Google to make PyTorch work nicely with TPUs. Some players like Anthropic started using JAX for training but all the small players are using Nvidia, I'm guessing it has something to do with Nvidia partnering aggressively with startups.

ismailmaj··on OpenAI Is Preparing to File for an IPO Soon
Their moat is cuda and cuda libraries and everything built on top.

When a new architecture drops, it's always PyTorch running on CUDA, other PyTorch backends are best effort, even if they reach feature parity, many industry power users went closer to the metal to squeeze performance and that stuff is too specific to Nvidia stuff.

if there is something that will beat Nvidia, it won't be something reaching feature parity with slightly better economics (like AMD, also Nvidia could just reduce their margins), it needs to be a novel approach worth rewriting the codebase for (maybe Cerebras, maybe a new player).

ismailmaj··on Meta's embrace of A.I. is making its employees miserable
I'm thinking that personally, technology is not bad in a vacuum and not necessarily bad in society, but it just reveals that our system is ill-equipped to guarantee good usage of it.

We could have fun defining what's good usage but we're so far from it, it would just make me sad.

ismailmaj··on Cerebras S-1
Unclear if it's the only cause but wafer scale is great for very low latency, but loses to throughput per dollar compared to classic Nvidia like GPUs. I don't think they can reduce the gap, SRAM is just more expensive than HBM and their architecture needs a lot of it.

So, the price makes it necessarily niche to some specific use-cases like HFT or intelligent duplex voice assistants, I'm still semi-bullish personally.

← PreviousPage 2 of 5Next →