HNHacker News
TopNewBestAskShowJobs

szvsw

898 karma · joined May 24, 2024

email: $hn_handle [at] mit [dot] edu

meet.hn/city/us-Boston

submissionscomments
szvsw··on Ara Agui Nakajima DCT compression algorithm
> so residual error derivatives in a sense?

Yep, loss = residuals*2 + lambda*sum(|a_i|), and we take the gradient of that with respect to the a_i to guide our gradient descent steps.

szvsw··on Ara Agui Nakajima DCT compression algorithm
Not quite - see my response for to GP for what I was describing.
szvsw··on Ara Agui Nakajima DCT compression algorithm
Not quite, though very related and I believe both should end up with essentially the same result.

Matching pursuit is essentially a greedy algorithm, if I recall correctly - please do correct me if I am wrong, where you conceptually find the component that explains the most data at each iteration, remove it, and then repeat the process on the residual data. Pardon if that isn’t quite the right explanation, but it’s what my intuition is recalling right now…

What I was describing was a simpler algorithm that can be done with gradient descent or any other vanilla optimizer.

Your model parameters are the coefficients a_i over all basis functions in the frequency domain representation. Run them through the synthesis function to get a time domain signal, and select the values of the time domain signal where your target data is known. Compute the squared error at each of those locations, and take the mean. This is your reconstruction error, and should be trivially differentiable with respect to the coefficients a_i. Compute an additional error term which is a standard L1 regularization, ie sum(|a_i|), which can be added to the reconstruction error term with some weight λ (λ=1 is even fine here, at least for simple problems), and then is also trivially differentiable (provided you haven’t initialized any of the coefficients to 0). As with any L1 regularization term, the resulting solution should be sparse in the L1 regularized parameters (look up visualizations of problems with only 2 model parameters to see how this emerges from the L1 contour lines of equal loss forming “diamonds” with the points on the axes).

szvsw··on AI-designed chips are so weird that 'humans cannot understand them'
There’s a few very cool examples where someone recently used RL to solve trackmania, and ends up having to add all sorts of constraints/penalties to prevent extremely strange exploits/glitches that are discovered IIRC… been a while since I watched.

https://youtu.be/Dw3BZ6O_8LY?si=VUcJa_hfCxjZhhfR

https://youtu.be/NUl6QikjR04?si=DpZ-iqVdqjzahkwy

szvsw··on Building a BitTorrent client from the ground up in Go (2020)
As someone who stopped torrenting around 2015 or maybe 2016, what happened to uTorrent? Also RIP what.cd, I still am sad that it is gone.
szvsw··on Ara Agui Nakajima DCT compression algorithm
The first time I used L1 regularization to recover a sparsely sampled signal’s DCT coefficients my mind was blown. If you’ve never tried it out, you should, it’s pretty crazy! The power of priors and choosing the appropriate inductive bias…

The classic learning example is to take a couple of sine waves at high frequencies, add them together, sample the summation at a very low frequency (so that the signals are way above the nyquist sample rate) or better yet just randomly at a handful of points, and then turn it into an optimization problem, where your model parameters are the coefficients of the DCT of the signal; run the coefficients through the inverse transform to recover a time domain signal, and then compute a reconstruction error at the locations in the time domain representation for which you have observed data, and then add an L1 penalty term to request that the coefficients be sparse.

It’s quite beautiful! All sorts of fun more advanced applications but it’s quite awesome to see it happen before your own eyes!

My personal signals & systems knowledge is pretty novice level but the amount of times that trick has come in handy for me is remarkable… even just as a slightly more intelligent data imputation technique, it can be pretty awesome! The prof who taught it to me worked at Bell Labs for a while, so it felt a bit like a guru sharing secrets, even though it’s a well documented technique.

szvsw··on Vine: A programming language based on Interaction Nets
Related article I’m enjoying for more context on interaction nets:

https://wiki.xxiivv.com/site/interaction_nets.html

szvsw··on Vine: A programming language based on Interaction Nets
Have you thought about how autodiff can be baked into the lang? Given that it is already seemingly all representing computation with graphs of operations, building AD into it seems like a natural extension that could open up some pretty useful applications/bring in some scientific computing interest.

I’m also curious to think about how the “time travel” might interact (if at all) with (1) basic algebra and (2) building off that, systems of equations and (3) building off of that something like implicit/crank-nicholson finite difference schemes.

Without having thought about it much, it feels intuitively like there might be an elegant way of expressing such algorithms using this notion of information traveling backwards, but perhaps not.

As an aside, although I’m not really a fan, the representation of time travel in Tenet seems like a natural analogy to include in your docs for fun if desired. More relevant, talking about things like backprop in standard ML packages might be one way of providing a shared reference point for contextualizing information flowing backward through a computational graph.

Finally, echoing what others have said about not burying the lede in the docs, as well as wanting to see more motivation/examples/discussion of interaction nets.

szvsw··on Malaysia is betting on data centers to boost its economy
Yeah cooling towers seems like the obvious choice, thanks for confirming! I’m sure there are some other more exciting designs that hook datacenters up to prosumer/mesh topology ambient-temp district heating/cooling loops - or there should be if there aren’t! - but what you described makes perfect sense.
szvsw··on Malaysia is betting on data centers to boost its economy
What about heat transfer? At least in a moderate, dry climate (whereas SP is hot/humid) there’s a natural benefit to having a high surface area to volume ratio, favoring wide/flat spaces, from the perspective of heat rejection. I’m not sure to what extent space cooling is actually a controlling factor in DC design, or what the typical strategies are (for instance I assume there are lots of new systems which make use of geothermal cooling). Anyways, just curious if you have any insight on that.
szvsw··on Diablo hackers uncovered a speedrun scandal
> You cited the exploiting as the one and only reason that cheating is interesting here

As far as I can tell, I did not… this is what I said:

> it’s very interesting to see how a practice that is fundamentally about exploiting generally unintended mechanisms within a system design still develops its own notion of ethics, legitimate and illegitimate exploits, etc

Removing extra clauses:

> it’s very interesting to see how a practice […] develops its own notion of ethics, legitimate and illegitimate exploits, etc

szvsw··on Diablo hackers uncovered a speedrun scandal
Runners on 2B are allowed to try to steal signs if they can crack whatever set the pitcher and catcher have switched to (well they use pitch com now); the astros used live video feeds to pick up the signs even with no one on base as well as computer assisted methods to crack the codes, and then related the pitches to the batter via the dugout.
szvsw··on Diablo hackers uncovered a speedrun scandal
> This is not at all the same thing as the Astros scandal. Astros didn’t break the agreed upon rules.

Sheesh, didn’t expect to re-litigate WS‘17 on HN today… I concede that there was at least some gray area in the codified official rules and the 2001 directive around electronic transmission, but it very clearly was a violation of the norms, IMO, and the Giants-Dodgers binocular incident in the 50s and its reception is good precedent.

> There is no “own notion” of ethics here, and it’s presumptuous and incorrect to suggest that exploiting games during a game exploit competition somehow reflects on ethics and makes the cheating “interesting”.

That’s not what I was suggesting was interesting to me. What interests me is the social phenomenon that is what the actual members of the community deem to be acceptable or in this case unacceptable, how the the community works to identify violations of norms, and then handle with the fallout.

szvsw··on Diablo hackers uncovered a speedrun scandal
> It's not a particularly complex or surprising "ethics" though

Yes, you are mostly right, but it’s still interesting (and positive!) that the community landed on an “anything goes so long as you are honest” set of norms.

As someone without much (well, any) interaction with speedrunning communities, it’s still fun to see how other internet communities have codified or tacitly established their operating procedures, expectations, norms etc.

szvsw··on Diablo hackers uncovered a speedrun scandal
> Speed running communities generally don't care what exploits you use, as long as you are up-front and honest about it.

That is, precisely, a set of ethics!

> There really isn't an "ethics" of what exploits are allowed or disallowed.

It sounds like there is: disclosed exploits are allowed, while non-disclosed exploits are disallowed. This is very clearly a set of ethics. It is different than that of industry, where generally speaking trade-secrets and non-disclosure of methodology is often considered critical to business success, short of establishing protected IP; in academia, disclosure is putatively the norm but there is plenty of partial disclosure of methodology rather than complete… etc.

> The problem isn't the part about what exploit was used; it's the part where he lied to the community about which category it should be classified.

That puts it very precisely, and also highlights his unethical behavior in the context of speedrunning’s norms around ethical representation of achievements.

szvsw··on Diablo hackers uncovered a speedrun scandal
As someone with absolutely no knowledge of speedrunning community norms, it’s very interesting to see how a practice that is fundamentally about exploiting generally unintended mechanisms within a system design still develops its own notion of ethics, legitimate and illegitimate exploits, etc. Not that different than something like the Astros technology-assisted sign-stealing scandal (as opposed to legitimate sign-stealing) against the Dodgers in the ‘17 WS, at least from the perspective of needing to identify the lines between fair play and illegitimate advantage.

Edit: children have helped point out the precise notion of ethics - no exploit is unethical on its face, so long as it is disclosed, which allows the speedrun to be correctly classified and compared to others in its class.

szvsw··on Complex dynamics require complex solutions
Doom is not quite the right word, at least in the apocalyptic connotation… it is too eventful, punctual, eschatological, teleological, suggesting a particular point of no return. Especially when combined with the word horizon, which suggests a point we can’t see beyond, a climatic singularity. Uncharted system dynamics? Cascading failures? Suffering? Wet bulb death zones? Crop failures? Another pandemic? Population collapse? Yes yes yes yes yes maybe maybe. But not a doom horizon. So fatalistic a term… does it simply function to absolve ourselves of the chilling experience of actually taking the time to think through what the world looks like beyond “the horizon” - absolution we seek simply because the truth is it will be terrible, but only incrementally so, and in a recognizable rather than unrecognizable way.

We can attempt to think through how things will play out, given what we already see happening and simple assumptions about power dynamics. People will live through it all, especially the wealthy (cutoff threshold: unknown; or rather, where is the elbow/knee between suffering and resilience? I don’t know) - some of whom will experience some degree of climate shock (cf. the palisades, Malibu etc for a recent example) but will largely find ways to continue exploiting their advantages to survive, even thrive, while others will suffer the incrementally normalized dangers of simply being alive, like frogs slowly boiling in someone else’s kitchen. People will keep fucking, having kids, trying to work shitty jobs, trying to work fun jobs, doing science and research, making music and bad reality television shows and brilliant novels and bad novels, crossing continents for better lives (whether that’s economic opportunity or merely to escape climatic incompatibility with life) while others resent them for it, and people will continue to post on Hacker News long diatribes lamenting the state of things.

I do think it is accurate to say we are *doomed* to this in the sense of being *fated* to it though.

I guess what I am saying is that to call what will happen a doom horizon is just a hood we put over our own eyes because it is too upsetting to really think with nuance about the most probable trajectory.

szvsw··on TeX and Typst: Layout Models (2024)
Right, I guess that’s my point: If Typst wants to compete with LaTeX, IMO it needs some sort of mechanism by which journals will deem a Typst submission acceptable, along with readily available templates for said submissions. That’s a big hill to climb probably, but probably the single most valuable development they could achieve from a product diffusion perspective.
szvsw··on TeX and Typst: Layout Models (2024)
> I think one of the best ways to overcome the enormous momentum of TeX is to point out its limitations (while still keeping an eye on Typst's limitations), and explain how Typst overcomes them.

One of the other easy ways to overcome it is to provide as many templates as possible for journals. I’ve used LaTeX for years, but would by no means consider myself an expert in LaTeX, as I’ve almost exclusively been able to grab a template from a journal or from my university, and then just draft in the relevant blocks, write equations, add figures, and, rarely, add a package. I would guess that there are a huge amount of LaTeX users like me out there. I do all my drafting on Overleaf. I love TeX (and curse my PI whenever he requires that we use Word/365 instead of LaTeX/Overleaf)… but so much of the benefit, for me at least, comes from the fact that templates are readily available for any journal I would want to submit to; my masters thesis was built in a template provided by my university; etc. I don’t have to deal with any of the cognitive overhead of styling and formatting (except for flowing the occasional figure) and can just focus on drafting.

For me to even consider typst, it’s pretty much a requirement that there is some degree of template parity actively being worked on. The most natural way to approach that would be to just sort every journal by impact factor and start working top to bottom; given that so many journals share templates due to being within elsevier, springer etc, it should be straightforward to reach a reasonable degree of parity relatively quickly.

Getting the major publishers to support and offer their Typst templates would make me try it out immediately for what it’s worth.

szvsw··on Do-nothing scripting: the key to gradual automation (2019)
It’s an interactive slide show, but it also has forced the user into the CLI context, where they are likely to already bring a different type of attention and focus than if they were reading the exact same set of instructions in a markdown file somewhere in the org’s repos. Not necessarily better or worse, just different due to simulating the experience of a CLI tool as a mechanism for prompting the user to take certain actions.
szvsw··on Introducing deep research
> the average person on the street will confuse Deepseek with Deep Research.

That’s probably a feature not a bug (from OpenAI’s perspective…).

szvsw··on Recent results show that LLMs struggle with compositional tasks
The point made by a commenter a few levels up questioned whether 98% reliability for adding two 100-digit numbers together should be considered noteworthy/acceptable etc.

My point is that assessing 98% reliability qualitatively depends on whatever kind of baseline system you are comparing against.

Obviously this reliability (and speed) of an LLM with 98% accuracy is atrocious compared to any basic automated arithmetic computation system. If you need to reliably add together 100 digit numbers, of course an LLM is a bad choice.

However, comparing it to the general class of systems which an LLM belongs to - ie “things that can add 100-digit numbers together quickly and also tell you about this history of the Ottoman Empire and a bunch of other stuff too, all within a relatively short amount of time” - this reliability is (potentially) impressive. We don’t have many such systems - a moderately educated human; a generic bespoke piece of software which process natural language requests and attempts to decide whether or not to retrieve information from a database/the internet or write and execute code, but could conceivably have been written in a crude manner 5 years ago; and LLMs (which might also take advantage of tool use as suggested in the previous example).

In this context - comparing to other systems with similar capabilities - the 98% reliability might be considered impressive, especially when you consider that it likely is much higher for more common simpler arithmetic problems.

My intention in bringing up “an average HN user” was to bring up an example of a reasonable benchmark to compare to, in my opinion. A HN user is a stand-in for a moderately educated person, which could reasonably respond to a variety of generic natural language requests in a short amount of time. My point was simply that adding together 100 digit numbers reliably and quickly while also being able to chat about Shakespeare or Kalman filters or planning a wedding is likely a more difficult task than we tend to give credence to, due to our familiarity with specialized systems which do arithmetic extremely well and extremely quickly.

szvsw··on Recent results show that LLMs struggle with compositional tasks
At pace though? What would your throughput be to maintain 98% reliability? How long do you think one execution would take? my guess would be between 30s-1m for a single summation of two 100 digit numbers? So let’s say you want a sample size of 200 operations to estimate reliability… maybe something like 2-4hours to get through the computations working non stop? That’s actually a nontrivial amount of focus on a highly repetitive task, which isn’t easy for a lot of people. I’m now genuinely curious what the average HN user’s reliability and throughput would be for adding together two random hundred digit numbers working as fast as they can.
szvsw··on Recent results show that LLMs struggle with compositional tasks
Can you add 100 digit numbers reliably in a short amount of time over a large sample size? How about without a piece of paper and a pen? Can you even remember a 100 digit number? It’s likely quite difficult…

This is not about comparing to reliability “of a computer” but rather to a process which responds to a wide array of natural language queries (both human processes an algorithmic processes)… and obviously the reliability goes up once tool use is included.

szvsw··on Machine Learning in Production (CMU Course)
Arguably that’s a separate (obviously critical) concern. I think it’s worth it to abstract that away as just a step that exists in the pipeline with its own set of concerns/challenges/methods etc that really requires its own deeper study to do well.

For instance, my ML work is almost entirely in the context of engineering simulation regression/surrogate development, where data quality/cleaning is almost no issue at all - all of the work is on the dataset generation side and on the model selection/training/deployment side.

Every job is different!

szvsw··on Stargate Project: SoftBank, OpenAI, Oracle, MGX to build data centers
AWS GovCloud already exists FYI (as you hinted) and it is absolutely used by the DoD extensively already.
szvsw··on Is the world becoming uninsurable?
A lot of what you said is intuitively/directionally correct, but misses a lot of important physics related to heat transfer in buildings and operational questions of space heating equipment.

This is your most accurate/relevant point:

> All of this is ignoring the fact that it’s easy to create a tiny personal heated environment around an individual (it’s called a woolly jumper).

Whereas this is plainly wrong:

> It’s much easier, and consumes less additional energy, to heat an occupied space, than to cool it.

And then the following is correct but the marginal reduction in load is minimal except in relatively crowded spaces (or spaces with very high equipment power densities):

> Thanks to the fact that your average human produces 80W of heat just to stay alive.

The truth is it is generally easier to cool not heat when you take into account the necessary energy input to achieve the desired action on the psychrometric chart, assuming by “ease” you mean energy (or emissions) used, given that you are operating over a large volume of air - which does align with your point about the jumper to be fair!

Generally speaking, an A/C uses approx. 1 unit of electricity for every 3 units of cooling that it produces since it uses heat transfer rather than heat generation (simplified ELI5). It is only spending energy to move heat, not make it. On the other hand, a boiler or furnace or resistance heat system generally uses around 1 unit of input energy for every 0.8-0.9 units of heating energy produced. Heat pumps achieve similar to coefficients of performance as A/Cs, because they are effectively just A/Cs operating in reverse.

Your point about a jumper is great, but there are local cooling strategies as well (tho not as effective), eg using a fan or an adiabatic cooling device (eg a mister in a hot dry climate).

> So cooling a living space is always more costly than heating a living space.

Once you move to cost, it now also depends on your fuel prices, not just your demand and system type. For instance, in America, nat gas is so cheap, that even with its inefficiencies relative to a heat pump, if electricity is expensive heating might still be cheaper than cooling per unit of thermal demand (this is true for instance in MA, since electricity is often 3x the price of NG). On the other hand, if elec is less than 3x the cost of nat gas, then cooling is probably cheaper than heating per unit of demand, assuming you use natural gas for your heating system.

szvsw··on Ingrid Daubechies awarded National Medal of Science
Daubechies wavelets are such incredibly strange and beautiful objects, particularly for how deviant they are compared to everything you are typically familiar with when you are starting your signal processing journey… if it’s possible for a mathematical construction to be punk, then it would be the Daubechies wavelets.
szvsw··on Programming as Theory Building (1985) [pdf]
Yep, this makes a lot of sense.

In general, we already have plenty of mechanisms for specifying interfaces/api specs, tests, relationships, etc in a declarative but more formal manner than natural language which probably all work , and I can only imagine we will continue to see the development of more options tailored to this use case.

szvsw··on Programming as Theory Building (1985) [pdf]
Do you have any repos or examples you can share? Would love to see an example of that in action!
← PreviousPage 3 of 10Next →