HNHacker News
TopNewBestAskShowJobs

jychang

3,006 karma · joined October 23, 2012

submissionscomments
jychang··on Claude Skills are awesome, maybe a bigger deal than MCP
That's the kind of thing LLMs would HELP with, though.

Comments may go out of date, but LLMs can quickly generate comments that are up to date. LLMs are more likely to prevent the situation that you described.

> In many cases AIs will even add comments that only make sense as a response to my prompt and don't make any sense in-context.

Yeah, this is a common issue to be fair.

jychang··on Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system
That's only if you're expecting 10 GPUs in use. They're dealing with ~1 GPU in use for a model, just sitting there. Alibaba has a very long tail of old models that barely anyone uses anymore, and yet they still serve.

Here's a quote from the paper above:

> Given a list of M models to be served, our goal is to minimize the number of GPU instances N required to meet the SLOs for all models through auto-scaling, thus maximizing resource usage. The strawman strategy, i.e., no auto-scaling at all, reserves at least one dedicated instance for each model, leading to N = O(M)

For example, Qwen2 72b is rarely used these days. And yet it will take up 2 of their H20 gpus (with 96GB VRAM) to serve, at the bare minimum, assuming that they don't quantize the BF16 down to FP8 (and I don't think they would, although other providers probably would). And then there's other older models, like the Qwen 2.5, Qwen 2, Qwen 1.5, and Qwen 1 series models. They all take up VRAM if the endpoint is active!

Alibaba cannot easily just timeout these models from VRAM, even if they only get 1 request per hour.

That's the issue. Their backlog of models take up a large amount of VRAM, and yet get ZERO compute most of the time! You can easily use an easing function to scale up from 2 gpus to 200 gpus, but you cannot ever timeout the last 2 gpus that's serving the model.

If you read the paper linked above, it's actually quite interesting how Alibaba goes and solves this problem.

Meanwhile on the other hand, Deepseek solves the issue by just saying "fuck you, we're serving only our latest model and you can deal with it". They're pretty pragmatic about it at least.

jychang··on Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system
Tensor parallelism, so you only need to store a fraction of kv cache per gpu.
jychang··on Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system
They definitely won't idle out- if they idle out, it'll take on the order of up to 60 seconds to load the model back into VRAM, depending on the model.

That's an eternity for a request. I highly doubt they will timeout any model they serve.

jychang··on I wish SSDs gave you CPU performance style metrics about their activity
You can try running LLMs on your own computer!

They have extremely distinct sounds coming from the GPUs. You can hear the difference between GPT-OSS-20b and Qwen3-30b pretty easily just based on the sounds that the gpu is making.

The sound is being produced by the VRMs and power supply to the GPU being switched on and off hundreds of times per second. Each token being produced consumes power, and each attention and MLP layer consumes a different amount of power. No other GPU stress test consumes power in the same way, so you rarely hear that sound otherwise.

jychang··on Titan submersible’s $62 SanDisk memory card found undamaged at wreckage site
If you read the article, the SD card was placed there by the camera manufacturer and then the device was sealed so it would withstand pressure, and then sold to divers. Blame the camera manufacturer's engineers.

Seems like the SD card of all things performed just fine, so it hardly seems like the weak point.

jychang··on Claude Skills are awesome, maybe a bigger deal than MCP
> have no problems with mechanics.

What? Shaq shot free throws at 85% in practice. Players with good mechanics like Curry wouldn't be caught dead shooting 85% during the season, he's always 90%+ at free throws. Curry would probably shoot free throws at 99.9% in practice; there's plenty of stories of him swishing 100 3-pointers in a row in practice.

I'm not saying Shaq would shoot free throws as good as "90% in game and 99.9% in practice" if he threw free throws underhanded, but clearly Shaq had mechanics issues.

jychang··on EVs are depreciating faster than gas-powered cars
It's just too small for most americans' tastes lol
jychang··on Claude Skills are awesome, maybe a bigger deal than MCP
I think the code comments straight up just help understanding, whether human or AI.

There's a piece of common knowledge that NBA basketball players can all hit over 90% on free throws, if they shot underhand (granny style). But for pride reasons, they don't throw underhand. Shaq just shot 52%, even though it'd be free points if he could easily shoot better.

I suspect there's similar things in software engineering. I've seen plenty of comments on HN about "adding code comments like a junior software engineer" or similar sentiment. Sure, there's legitimate gripes about comments (like how they can be misleading if you update the code without changing the comment, etc), but I strongly suspect they increase comprehension of code overall.

jychang··on Intel Announces Inference-Optimized Xe3P Graphics Card with 160GB VRAM
Slow is worse than nothing.

What's the point of this card that's going to be released around the same time as DDR6, and DDR6 will be faster? Might as well as use cheaper system RAM if you system RAM is slower than VRAM.

jychang··on Intel Announces Inference-Optimized Xe3P Graphics Card with 160GB VRAM
1. 600GB/sec is still slow as hell. You might as well as use regular DDR5 RAM then if you're so slow, you can spec regular system DDR5 RAM faster than 600GB/sec. The half decade old consumer 3090 is 1.5x that speed. The current 5090 is 1,792 GB/s. The current nvidia datacenter cards are 8 TB/s. What's the point of having lots of VRAM if your system RAM is faster?

For context: if you have a 160GB dense ML model in VRAM and you're just running 600GB/sec, you can do... roughly 4 tokens per second AT BEST. That massive amount of VRAM is unusable if it's slow.

2. 512 bit LPDDR5x is most likely just 512GB/sec with typical LPDDR5x that's not overly expensive. I would be HIGHLY surprised if they gave it the more expensive RAM that'd break 600GB/sec. The Intel B60 is at 456 GB/s and that's using GDDR6.

Honestly, you're better off waiting for regular DDR6 to come out in a year and just build a system using that.

jychang··on Claude Haiku 4.5
It’s not instantly fast though. Context is probably ~20gb of VRAM at max context size. That’s gonna take some time to get from SSD no matter what.

TtFT will get slower if you export kv cache to SSD.

jychang··on Intel Announces Inference-Optimized Xe3P Graphics Card with 160GB VRAM
It’s LPDDR5x

It’s gonna be slowwww

It’s gonna be what, 273GB/sec vram bandwidth at most? Might as well as buy an AND 395+ 128GB right now for the same inference performance and slightly less VRAM.

jychang··on Nobel Peace Prize 2025: María Corina Machado
Nope, "just placing a bet" is still illegal if you’re trading on material non-public information (MNPI). Even if you don’t disclose anything, U.S. regulators can treat a prediction-market wager as a derivatives trade made with misappropriated confidential info.

Prediction-market contracts are either CFTC or SEC territory. Many "event contracts" are regulated as swaps (CFTC). If a contract references a single security/issuer, then it falls under security-based swap rules (SEC). Either way, trading them with MNPI risks liability under the applicable anti-fraud rules. (CFTC Rule 180.1 is guided by SEC 10b-5 case law on misappropriation.)

The CFTC uses the misappropriation theory: trading while breaching a duty to keep information confidential is prohibited, even if you never "tip" anyone. They’ve brought cases on this theory. CFTC enforcement and guidance explicitly cite misappropriation cases (Ruggles 2016, Motazedi 2015, EOX, etc) as "insider trading" analogs for non stock trading.

> They still called it a market.

I'm saying you can pivot your hypothetical company to a website-that-pays-for-news-like-TMZ. Last time I checked, they don't call TMZ a market.

jychang··on Ask HN: Abandoned/dead projects you think died before their time and why?
They stopped making the Chromecast hardware
jychang··on Liquid Glass Is Cracked, and Usability Suffers in iOS 26
He's being sarcastic, "bad" and "bad ass" are clearly opposites.
jychang··on Nobel Peace Prize 2025: María Corina Machado
> I don't believe that insiders are actually that discouraged from trading even on corporate markets, and more importantly I don't believe the public believes insiders are prevented from trading.

Yeah, you're just describing humans behaving irrationally then. That's clearly not rational behavior (which to be fair, is totally expected for humans), which means that even if it's descriptive of the current markets now, that's not actually ideal.

jychang··on Nobel Peace Prize 2025: María Corina Machado
> Why would we protect willing participants betting on arbitrary events?

Because some information affects the stock market. So Regulation FD already applies, for financial news.

> Even if we ban on this one too

The UK already banned this, see other comment above in this thread.

> insideinformationverywelcomemarket

You might as well as cut out the middleman, don't call it a "market" anymore, and just call it "crowdfunded-will-pay-reward-for-insider-information-website". Or, basically a crowdfunded TMZ. TMZ will pay thousands of dollars for non-financial info on celebs and then publish that news. That that point, you're just describing a slightly classier crowdfunded TMZ.

jychang··on Nobel Peace Prize 2025: María Corina Machado
> Emperically, we have plenty of participation by the public in prediction markets without a prohibition on insider trading.

That's not true. As mentioned in this thread, the UK has laws in general against this (for betting). The USA also already has laws against specifically insider financial disclosures (Regulation FD) for corporations, that also applies to betting on prediction markets (Dirks vs SEC ruling in the general case). I'm not sure if Regulation FD applies to Nobel prizes, though, as that's not regulated by the SEC, and I'm not sure if the USA has general laws on illegal betting. But therefore, I do not think you can claim that "there is no prohibition on insider trading", as that's already clearly mostly illegal and thus already priced in for the public participation in prediction markets.

> It's exactly the opposite. Banning insider trading in prediction markets would be a nearsighted way to increase volume (by encouraging more public participation) at the cost of long term value (accuracy, because insiders have the accurate information).

Again, the current laws already ban insider trading from corporate sources, so the status quo is already what I propose; you don't see insiders trade on predictions like "would OpenAI release GPT-5 in 2025" as that's against the law.

> Prediction markets can only be an "information value source" to the extent that they are accurate.

This is also not true. Bayes Theorem! The information does not need to be fully accurate, just more accurate, enough to update from P(A) to P(A|B) given P(B), where P(B) is the prediction market's price on a certain prediction! That means an inaccurate prediction market can still inform your knowledge updates, if you can derive information from it.

jychang··on Nobel Peace Prize 2025: María Corina Machado
Smart people are financially hurt by it.

Isn't that the entire point of the prediction market model? You derive the wisdom of the crowd by enabling smarter people who make better educated guesses as predictions (from the same access to information as others) to win out over time. They are incentivized to do this financially, by winning bets.

If markets just becomes "first person to cheat takes all", then there's no incentive to NOT cheat. So you drive away the people who power the prediction market in the absence of information. Over time, this means that the prediction market is just random noise until a leaker publishes news. At that point, you might as well as just skip the middleman and just make a website "LeakNews" where you can directly offer bounties that goes towards a reward paid out to a leaker for news you are interested in.

jychang··on Liquid Glass Is Cracked, and Usability Suffers in iOS 26
Yeah, that's the funny thing about this article title.

Apparently the author (and most commenters here!) do not realize "cracked" is VERY popular gen z slang for being very good at something.

https://www.urbandictionary.com/define.php?term=Cracked

jychang··on Nobel Peace Prize 2025: María Corina Machado
Participation by the general public is an important part of prediction markets! That's the entire point of the wisdom of the crowd. Before an insider tilts the market in one direction, in the absence of much insider information, prediction market advocates point out that the prediction market will still capture information value that you can't glean from just one source.

If you discourage public participation in prediction markets "oh I will just lose my money to an insider a few hours or days before the final result, so why bother", then the end result is that nobody participates until an insider makes a big bet. Then the market is worthless until the insiders jump in. Is that really what prediction market advocates want?

The other point is that:

> Prediction markets don't provide capital for productive businesses

Is not true. There's more than just capital in terms of cash. There's human capital (employees), brand value... and importantly, information. Which is what prediction markets intend to do in the future: become an information value source for productive businesses.

Money isn't the only unit of value.

Allowing insider trading seems like a nearsighted way to increase volume for prediction markets, at the cost of long term value.

jychang··on Nobel Peace Prize 2025: María Corina Machado
This isn't illegal but feels like it should be illegal. You don't see corporate officers trade their stocks right before a big announcement, because there are laws on it.

Other than the fact that polymarket is legally not a stock market, what really is the difference between insider trading on a stock market vs insider trading on polymarket? Does anyone have a good argument for why one should be illegal while the other is legal?

jychang··on After nine years of grinding, Replit found its market. Can it keep it?
You don't write for your median reader, you write for the vast majority of your readers.

That's a basic concept of writing. Journalism should be accessible, so even if you know what a database is and how to deploy it in different envs, you shouldn't write assuming that. If a large portion of your readers don't know what you're saying, you've failed as a writer. If your readership includes high school students, you write with that as the baseline.

Richard Feynman certainly didn't write as if he assumed the reader knew particle physics. Be like Richard Feynman.

jychang··on N8n raises $180M
It got viral on Tiktok of all places a while back, so I'm not surprised at the number of stars.
jychang··on Memory access is O(N^[1/3])
The empirical argument actually states that memory access is O(n^1/2)

https://www.ilikebigbits.com/2014_04_21_myth_of_ram_1/3_fit.... "The blue line is O(√N)."

This has been rehashed many times before, and the best blog post on this topic is here: https://www.ilikebigbits.com/2014_04_21_myth_of_ram_1.html

jychang··on Memory access is O(N^[1/3])
I mean, if you want to talk about our actual tech, it's bound by lithography of silicon chips, which are largely n^2, on printed circuit boards, which are n^2, on the surface area of the earth, which is n^2.

This has been observed since at least 2014: https://www.ilikebigbits.com/2014_04_21_myth_of_ram_1/3_fit....

Pretty much every physical metric pegs memory access as O(n^1/2).

jychang··on Memory access is O(N^[1/3])
It's obviously about latency. How do you not see the latency aspect of it?

Latency is directly bound by the speed of light. A computer the size of the earth's orbit will be bound by latency of 8 minutes. A computer the size of 2x of earth's orbit will be bound by a latency of 16 minutes, but have 4x the maximum information storage capacity.

The size of the book is directly proportional to the speed of light. Ever heard of ping? Does the universe you live in have infinite speed of light, and therefore you don't see how R contributes to latency?

jychang··on Memory access is O(N^[1/3])
> infinite-time or infinite-memory computer

That doesn't apply for the Bekenstein Bound though.

Literally the first line of the wikipedia article:

> In physics, the Bekenstein bound (named after Jacob Bekenstein) is an upper limit on the thermodynamic entropy S, or Shannon entropy H, that can be contained within a given *finite* region of space which has a *finite* amount of energy—or equivalently, the maximum amount of information that is required to perfectly describe a given physical system down to the quantum level.

jychang··on Memory access is O(N^[1/3])
Strong disagree- this is a fundamental core aspect of information theory.
← PreviousPage 7 of 28Next →