HNHacker News
TopNewBestAskShowJobs

szvsw

898 karma · joined May 24, 2024

email: $hn_handle [at] mit [dot] edu

meet.hn/city/us-Boston

submissionscomments
szvsw··on Combining 15s interval whole-sky-camera photos to form a 4y spanning keogram
Highlight related to an analemma: the figure the sun traces if you make a parametric plot of its position in the sky at fixed time T as a function of the day of the year d, ie f_T(d) = (azimuth,elevation).

https://solar-center.stanford.edu/art/analemma.html

szvsw··on Programming as Theory Building (1985) [pdf]
I mostly agree with what you were saying, but I don’t think I was advocating for “no code” entirely, and certainly not the elimination of engineers entirely.

I was trying to articulate the idea that code generation tools will become increasingly sophisticated and capable, but still be tools that require operation by engineers for maximal effect. I see them as just another abstraction mechanism that will exist within the various layers that separate a dev from the metal. That doesn’t mean the capabilities of such tools are limited to where they are today, and it doesn’t mean that programmers won’t need to learn new ways of operating their tools.

I also hinted at it, but there’s nothing to say that our orchestration of such systems needs to be done in natural language. We are already skilled at representing procedures and systems in code like you said; there’s no reason to think we wouldn’t be adept at learning new languages specialized for specifying higher order designs in a more compact but still rigorous form to codegen systems. it seems reasonable to think that we will start developing DSLs and the like for communicating program and system design to codegen systems in a precise manner. One obvious way of thinking about that is by specifying interfaces and test cases in a rigorous manner and letting the details be filled in - obviously attempts at that now exhibit lots of poor implementation decisions inside of the methods, but that is not a universal phenomenon that will always hold.

szvsw··on Programming as Theory Building (1985) [pdf]
> without a human going through the necessary thought processes and problem solving steps to build the theory of the software as described in this paper

We might not be there yet (well we definitely are not) but it does not seem out of the question that within a generous 10 years we will have systems which can leverage graphs, descriptive language, interpreters, and so on to plan out and document and iterate and refine the structure of a problem and its architectural solution in tandem with developing the solution itself iteratively at a very effective level, given a sufficient explanation of the goals/problem - or more importantly/phrased another way, following the initial theory of a problem formulated by the human; the kind of documentation produced by such systems can also be more easily ingested by other non-human systems, potentially remedying some of the challenges with outlining/documenting/transferring the theory of the problem that humans have.

And what prevents a human from doing code review on such a system’s outputs? Now maybe your point was that the simple expense of a human’s time is the barrier, especially given that you were talking about the context of companies using LLMs to speed up their code production (read: eliminate cost centers), but in that case the errors that may come from poorly designed procedurally generated codebases just reads like bad project management to me for which the chickens will ultimately come home to roost; the companies which can successfully integrate such procedurally codegen engines while still maintaining strong design principles, maintainability, simplicity, etc ought to outcompete their competitors’ slop in the long run, right?

Having said all that, I think the more important loss is that the human fails to build as much intuition for the problem space themself by not being on the ground in the weeds solving the problems with their own solutions, and this will struggle to develop their own effective theories of the problem (as indicated by the title of the article in the first place).

szvsw··on Programming as Theory Building [pdf]
> It seems to me that one consequence of the "Theory Building View" is that: instead of focusing on delivering the artifact or the documentation of said artifact, one should instead focus on documenting how the artifact can be re-implemented by somebody else. Or in other words optimise for "revival" of a "dead" programs.

Arguably, this is the entire spirit of academia, which mildly serves as a counter example, or at least illustrates the challenges with what you are describing - even in something where the stated goal is reproducibility, you still have a replication crisis. Though to be fair, I think part of the problem there is that, like you said, people focus too much on “documenting the artifact” and not “documenting how to produce the artifact,” but this is often because the process is often “merely” technical and not theoretical (and thus not publishable) despite being where most of the hard work and problem solving and edge case resolution and so on happened.

Edit: oh, and I would also mentioned, that the kind of comment you’ve described which focuses on why some process exists in the form it does to better explain how it does what it does aligns closely with Osterhout’s notion of a good comment in A Philosophy of Software Design.

szvsw··on PWCT2: Programming Without Coding Technology
I agree that typing probably is not the limiting factor in most cases and for most people, and so the conversation might seem a little silly… But that’s precisely because typing is fast. Once you know what you are doing, you start typing and you are done relatively quickly, and probably spent a lot of the time typing thinking about what you would do next… but we are specifically talking about a scenario where typing is the limiting factor, or even entirely infeasible, for a variety of reasons.
szvsw··on PWCT2: Programming Without Coding Technology
It really really depends on what your goals are and what type of design domain you are working in, eg jewelry, architecture (and what within architecture), fabrication, experimental sculpture, etc.

At the end of the day though… play. Lots of play! Playing within Grasshopper and just making crazy things is a great way to build intuition.

at the same time, it of course is very helpful to have directional play, ie problem solving, frequently in the form of trying to recreate - or parametrize/generalize/abstract - a specific pre-existing design.

It also helps a lot if you are pretty familiar with modeling in rhino (ie working with construction planes, curves, etc) since a lot of grasshopper worker is just a matter of formally capturing a sequence of operations that you might perform in Rhino into a codified DAG. Unfortunately some of the naming of rhino operations does not align with their corresponding grasshopper components (and the corresponding components might not perfectly align their api with the corresponding rhino commands) but developing that geometric intuition is essential.

There’s also some stuff that is ultimately more efficient to program within python / CSharp / Visual Basic within grasshopper so always keep that in mind… but it can become another layer of complexity (or simplification…) to learn at the same time.

szvsw··on PWCT2: Programming Without Coding Technology – Written in Ring
Yeah good points. You can easily imagine composing graph based/data flow programming languages with just your mouth, ie “create node:<type>:<name>; connect node:<name>:<outlet> to node:<name>:<inlet>” as a relatively easy way to synchronize a visual programming language’s traditional mouse-based construction engine to speech based construction with realtime visual updates etc.
szvsw··on PWCT2: Programming Without Coding Technology
Yeah, the high frequency input capabilities of hands/fingers on keyboard are pretty remarkable and hard to beat, especially in conjunction with navigation tools like vim and especially when you start considering things like stenography applied in a programming context (intellisense/copilot etc can in some ways be be considered a form of that already).

I suspect natural language speech is too slow, but i do sometimes wonder if the only other part of our body capable of such high throughput (and the requisite fine motor control) is our mouth/tongue… I’ve always wondered what a programming language optimized for spoken input would look/sound like.

Edit: mentioned this in another comment, but I think graph based visual programming languages would actually be pretty well suited to speech entry, especially with shorthands for common operations like “create node” := “kurr” or “connect” := “kuh”.

szvsw··on Overcoming Resistance to Extreme Programming
I can’t help but feel that every different flavor of formalization of the process of collaborative [software] development feels… nearly the same? At the end of the day.

Whether there is a self-conscious attempt to avoid being accused of dogmatics, or alternatively a fully embraced biblical dogma, it all feels like the same basic advice repackaged in new language. Sometimes the formalism is meant to create a framework to help the team organize better, but…

I feel like it all just comes down to:

- above all, be consistent but not too rigid

- talk with your colleagues, keep up to date with each other

- think about your constraints and goals frequently and check in with how you are relating to them

- work hard but not too hard

- have some systems, any systems in place, to check for poor work product and highlight good work product. What is good work product? Well, to use the famous quote, you’ll know it when you see it!

To be fair, some frameworks may obviously help some teams more than others, but like any tool evaluation and selection process, it always seems so highly contingent which process management method might work well or not, many might work equally well, and most of it comes down to the team and people more than it does the system…

I could be totally wrong as I don’t have much experience in teams or orgs beyond 5-10 people, but still… just my surface level understanding of all this.

I’m curious what others on here with far more experience and exposure to a variety of organizational frameworks have found and think about my assessment.

szvsw··on PWCT2: Programming Without Coding Technology – Written in Ring
There are many many many dedicated grasshopper modeling channels, both in the “watch and chill” style and the explanatory style. Once you’ve done lots of grasshopper programming and can recognize component icons from both the base library and the cornucopia of grasshopper plugins, you can skip through all of those videos at like 3x speed and very rapidly absorb the graph structure.

One of the most interesting parts of grasshopper is how components operate over tree based data structures - ie the equivalent of numpy broadcasting.

szvsw··on PWCT2: Programming Without Coding Technology – Written in Ring
While I agree with 99% of what you said…

> I’d say it’s a mouse-driven IDE with a YAML-like code. But in its current guise, I can see people itching to use the keyboard for speed once they know what text pieces they want.

It’s useful to keep in mind that a “mouse-driven” IDE could have some real benefits from an accessibility perspective. I agree that this generally might miss the mark from a variety of goals, especially compared to graph based dataflow programming languages (see my other comment in this thread), and to really maximize accessibility as a goal you would need to actually design it for that from the ground up, but it’s still good to see experiments with other modalities and raise the question. At the end of the day the fraction of programmers who have difficulty typing is low… but that doesn’t mean it should be ignored, especially considering that one day almost all of us on here will have trouble typing, but likely still have the urge to code!

szvsw··on PWCT2: Programming Without Coding Technology
Just going to highlight some of my favorite graph/flow based programming environments:

- Max/MSP - originally for audio/visual synthesis, but widely used for generative art (in the meaning of generative that existed before the AI boom of the 2020s)

- Grasshopper & Dynamo : structural/geometry/architectural generative design

- Modelica/Dymola/OpenModelica etc: symbolic equation modeling and diffeq solving, widely used in HVAC/systems modeling, automotive, aeronautical design

- Modular synthesizers - analog audio synthesis (and these days, digital as well!)

It’s interesting to see that almost all of the examples mentioned above leverage the fact that they are abstractions on top of lower level programming frameworks which are tailored to specific tasks in order to make fantastic, intuitive interfaces that are easy to learn and quick to iterate incredibly complex designs within their domains, while the example linked here is not really domain-specific and is much closer to traditional programming (or something like Scratch)

szvsw··on Zildjian, a 400-year-old cymbal-making company in Massachusetts
Percussion synthesis is a deep and wonderful practice within electronic music, especially techno and especially in its more experimental manifestations. I’ve never encountered anything specifically claiming to emulate specific cymbals but I’m sure it’s out there galore. In any case, cymbal and hihat synthesis is a classic task in techno production.
szvsw··on Zildjian, a 400-year-old cymbal-making company in Massachusetts
Don’t forget the Russian and Turkish Baths in the east village (NYC) - my favorite instance of a familial dispute resulting in a business splitting. Now it is two businesses occupying the same physical location, but different temporal locations: when you get a pass it’s only valid for certain days of the week depending on which brother’s business you purchased it for.
szvsw··on Zasper: A Modern and Efficient Alternative to JupyterLab, Built in Go
I actually think the problem you are describing is actually sometimes helpful from a design perspective, if you can be conscientious enough to periodically review your notebooks and figure out what is the actual useful code which should be properly integrated into the codebase vs what is the “one-off” / non-modular code. Like you mentioned, calculation vs side-effects is one way to help you decide but not the only. There’s definitely no single answer. The key is to just periodically figure out what ought to be refactored into library code, which notebooks should just be straight up deleted (hopefully as many as possible - you can always get them back in your git history if needed!), and so on.
szvsw··on Zasper: A Modern and Efficient Alternative to JupyterLab, Built in Go
I’m not adding a lot to the conversation, but it’s not often you run into someone who contributes to creating a tool so fundamental to your daily life, career, growth as a researcher etc, so let me just take the opportunity to say: thank you and the rest of your team for creating such an amazing interactive tool.
szvsw··on Hitting the Jackpot: The Birth of the Monte Carlo Method – LANL (2023)
Absolutely! I’ve gotten to work on some projects with folks from the labs - pretty much meaningless projects in the grand scheme of research going on there, but still feel proud to have done it and lucky to have worked with them.
szvsw··on Electrical circuits encased in fluid may reshape data-center design
> However, that doesn't solve the problem of what to do with the heat once you have it.

Unfortunately they are not usually in great locations for this since you need other buildings, but if you have an ambient temperature district heating and cooling system for buildings nearby (sometimes called a 5th gen DHC system [1] - mainly characterized by the notion of prosumers ie producer/consumers of heat connected to a single loop system, rather than a more traditional supply loop/return loop topology) then datacenters can very easily fit in to the mesh as producers. Temps are typically 10-20c for ambient loops. The main insight is that that operating range is one that can be used quite well for both extracting heat and dumping heat simultaneously at different locations in the network.

You can also of course just add the heat directly to a hot water supply loop, eg [2]

[1] https://en.m.wikipedia.org/wiki/District_heating (see the section in history on 5th gen)

[2] https://www.nrel.gov/computational-science/waste-heat-energy...

https://www.nrel.gov/docs/fy19osti/73644.pdf

szvsw··on Hitting the Jackpot: The Birth of the Monte Carlo Method – LANL (2023)
I always completely forget that the metropolis-hastings algorithm is named after someone whose last name is actually Metropolis.

It never ceases to amaze me what an environment Los Alamos was for producing so much foundational research.

szvsw··on Hitting the Jackpot: The Birth of the Monte Carlo Method – LANL (2023)
I think we are on the cusp of another renaissance in Bayesian / MC methods - specifically, I think in some relatively short timespan (5 yrs? 10 yrs?), it’s reasonable to think that some new algorithms that are massively parallel at their core will break through. Whether it’s on the VI side or MCMC side or BO/GP side something totally new I don’t know, but it just feels like it is bound to happen eventually.

Big +1 for that textbook!

Also giving a +1 to the Bayesian Optimization/Gaussian Processes textbook [1] that came out last year - I mean 2 years ago - beautiful graphics and full PDF officially hosted.

[1] Garnett 2023, https://bayesoptbook.com/book/bayesoptbook.pdf

szvsw··on Nvidia bets on robotics to drive future growth
Citing a specific (tragic) incident isn’t really great evidence in re: safety. You have to normalize by something like accidents/mile driven and compare to comparable services (taxi/uber etc) - having said that I couldn’t quickly find any sources either positive or negative on those stats (besides Waymo PR docs) so I’m not saying you’re necessarily wrong. just wanted to point out the obvious flaw with citing anecdotal evidence for something like this.

You could easily use the same logic to say humans haven’t solved driving yet either!

szvsw··on Beyond Gradient Averaging in Parallel Optimization
Very cool! Glad to hear my intuition is on the right track… I’m very much on the applied ML for engineering design side as opposed to the bleeding edge research side, so in terms of multi-node training I haven’t done much more than spin up a few GPUs and let PyTorch Lightning handle the parallelism, but cool to try to keep up with this stuff.

Thanks for the response and good luck with this!

szvsw··on Beyond Gradient Averaging in Parallel Optimization
Just skimmed the abstract and it immediately has me wondering if there are ways that this research intersects with explainability research - specifically, the notion that certain inputs only activate certain portions of a network. I wonder if at some point that kind of information could be leveraged to provide better sorting of datasets into batches. Obviously this conflicts a little bit with the notion that you want your batches to be just completely random permutations.

In some sense, if two inputs activate the same parts of the network at high intensity, then they are more likely to result in conflicting gradients if they are in separate microbatches from the same macrobatch.

I’m curious if you think long term, there could be some utility in trying to periodically extract information about how samples activate the network and use that (or some other representation) to better sort the micro/macro batches to maximize your parallelism by making conflicts less likely.

Obviously there would be some sort of time penalty dealing with calculating whatever that representation is and sorting so that overhead might far outweigh any gains, but I could see it as at least plausible that there might be some contexts/scales where making that periodic investment could pay off.

szvsw··on Can LLMs accurately recall the Bible?
It seems like LLMs would be a fun way to study/manufacture syncretism, notions of the oracular, etc; turn up the temperature, and let godhead appear!

If there’s some platonic notion of divinity or immanence that all faith is just a downward projection from, it seems like its statistical representation in tokenized embedding vectors is about as close as you could get to understanding it holistically across theological boundaries.

All kidding aside, whether you are looking at Markov chain n-gram babble or high temperature LLM inference, the strange things that emerge are a wonderful form of glossolalia in my opinion that speak to some strange essence embedded in the collective space created by the sum of their corpi text. The Delphic oracle is real, and you can subscribe for a low fee of $20/month!

szvsw··on Long Term Software Development
Thank you for the detailed feedback! Great insight and makes a lot of sense.

> Particularly in open source, you see remarkably little investment in performance engineering or scalability, so it is always straightforward to write your own implementation that is many times faster on the same hardware. Even basic things like popular data format conversion libraries are surprisingly consistently suboptimal. In most cases, the kinds of changes we'd want to make cannot be upstreamed to that library; most open source libraries heavily optimize for simplicity and the widest set of users to the exclusion of maximum performance.

Yeah, I have even encountered this myself once or twice - i.e. re-implementing some optimization/search algos, sampling processes, or numerical simulation methods to be more tightly coupled to our requirements/context for great performance gain - despite not being in a context where heavy performance engineering has become necessary yet. This obviously overlaps with your point about using only a tiny subset of library functionality.

> Another issue is the impracticality of threading fine-grained metrics and observability through a hodgepodge of libraries that were never designed to be used that way or to work together. This is a big deal if you want to heavily automate operations.

Yes, I personally have not ever run in to this because of the relatively small scales I am operating at but makes perfect sense. Same goes for the security concerns. Obviously these are more serious concerns for larger and longer timescale projects (as discussed in the original blog post).

> engineer-month here, a few engineer-months there

I've only worked in contexts where there are no more than 5 simultaneous active devs on a project, so the long-term investment for longer term gains is probably in higher tension with the short term needs. obvious red flag for increasing technical debt/deferred maintenance/downstream complexity, but still can be difficult to justify diverting 20% of team's total productivity for a long term investment when there are short term demands. But always good to be reminded that those long term investments shouldn't be rejected outright and should be more seriously considered.

> I also like that this forces a discussion of actual requirements for that functionality. Far too much software makes their requirements whatever a library offers. I arrived at this point gradually over many, many years. The practical maintainability and opportunity for optimization have been key. In far too many cases, it turns out that a library dependency could be replaced with a couple hundred lines of code that actually worked better and which matched the design and style of the rest of the code base. There is a learning curve but once you've written a common class of library once, it is straightforward to do it again.

This is a really excellent and concise way of summarizing it. I'm definitely going to try to push this with my team more often and now have a great resource for how to articulate it. Thanks again for taking the time to explain your perspective.

szvsw··on Long Term Software Development
Oh I have 100% run into dependency hell many times in the Python ecosystem! I am well aware. But the overall force multiplication of that ecosystem has far outweighed the cumulative day or two per year spent in dependency hell.

As someone who works in both academia (currently a PhD student!) and industry (at the aforementioned startup), it just seems obvious to me that the situation you are describing is less of an innate issue with incorporating dependencies and more an issue with the lack of training on environment management / containerization and lack of experience with software development in general that you see in academic projects. In a relatively competent professional shop, these sorts of issues in my experience crop up occasionally but there are so many mechanisms available to (a) mitigate them from appearing in the first place and (b) have rough plans or strategies in place ahead of time if they do crop up and (c) continuously address them when they do appear, which prevents it from ever blowing up into a true dependency hell. It seems like being judicious about when you decide to use dependencies and how you manage them is sufficient, as opposed to actively avoiding them altogether...

Anyways to me it definitely is the casethat Python has loads of footguns/risks in re: dependencies when someone is developing without thinking about dependency management/long term planning, which is usually the case with academic projects, but I don't think these dangers are so bad when the user is actively aware of them and plans for them - or not bad enough to make the risks outweigh the rewards of adopting the dependencies... it definitely means that the ecosystem is elevating risk levels, but I would also think that there are organizational/management issues which contribute significantly to (and compound with) the issues you've described in an academic context.

> My estimate is that the shelf-life of a project using just these three will not exceed 3 years in >90% of cases.

That seems reasonable; it's problematic from the perspective of the replication crisis in the sciences, but from a different perspective - if the code/project has no users or active developers for 3 years, at what point does it just become cruft? Is it okay to let code die? I don't love this view, but my PI strongly feels that no matter how excellently or poorly engineered a GitHub repository is, it doesn't matter from the perspective of sciences - the only thing that matters is if (a) the methodology is explained thoroughly enough in the literature you produce (in which case the repo can be trash and unrunnable in a few years - if the method is useful enough it will be copied and expanded on etc by others naturally or just completely independently, and 3-5 year old methods are out of date anyways) or (b) it goes on to be actively used by you, your peers, or people in industry, in which case the literature does not matter (but by definition the repo is not trash as it is in continuous use/active development/evolution). I think their are some flaws in this view but I also see merit in it.

Anyways, having said all of this, I hate it when one of my peers at the startup I work with wants to pull in some dependency for a relatively straightforward search algorithm they don't understand when I think it would be easier to just implement it ourselves, so I'm contradicting myself :)

szvsw··on Long Term Software Development
> I also find it difficult to rationalize third-party dependencies these days, they almost all become disappointments over the long-term. The benefits rarely justify the risks.

What kind of software development do you do? As someone who is in ML/data science and works primarily in python, occasionally TS/React, at a small company (about 5 FTE, one round of funding), that sentence is so completely foreign to me. Again, this might just be because I am working primarily on ML systems in Python on relatively small teams (also in academia at the same time), where the most basic building blocks like Pandas (or Polars), PyTorch, XGBoost, SKLearn, etc are so essential and so it’s pretty hard to imagine ever being at a scale where they would be feasible or worth it to replace with our own code… and anyways if support for libraries like those collapses, the only explanation is that the entire landscape has shifted so much that it’s hard to imagine the underlying tech of the company not also having naturally shifted with it…

Maybe I am misunderstanding how you are using “3rd party,” or maybe you are just operating in a completely different staffing and financial context which I don’t have experience with.

Just curious for more details about what kind of world you operate in where that philosophy is viable!

szvsw··on Apple must ensure interoperability of iPhone with rivals, says EU
I think you’ve articulated it pretty clearly here, but I still have minor quibbles. Back to the original point… I might be misconstruing it, but reads a bit like this to me: something should be free of consumer protections if consumers have cheaper alternatives; they chose the more expensive option, and therefore they should suffer the consequences of that good or bad; and furthermore, we shouldn’t care if the company selling the product (allegedly) actively works to make it harder for them to transition to the cheaper alternatives in the future should they so desire.

Does that seem like a misconstruction of the part of the argument that we originally started this string of comments on? I think there are other slightly better arguments against it but this one seems pretty flawed to me.

It seems reasonable to me that when a product hits some critical, hard-to-define adoption threshold, even if it is a “luxury” choice by your definition, there is still a good argument for placing consumer protections in place. It’s also reasonable to believe those actions would be nanny-state failings, stifle innovation, drive business away etc etc, but I do think the first point of view is reasonable.

szvsw··on Apple must ensure interoperability of iPhone with rivals, says EU
I guess I disagree that price alone makes something a luxury good. Is a 10 year old Honda civic a luxury good (typically around $10-13k) even though there are probably cheaper and comparable alternatives? I would say no? Not a perfect analog because I do think there is merit in your point that there are “actually cheap” contemporaneously manufactured phones (Samsung etc). but still, maybe it’s just semantic or pedantic but when 1.3b active iPhones are out there, to me that indicates it is something other than luxury and more importantly does cross a threshold where there are real reasons to consider larger anti-consumer patterns, even if the consumer is choosing the more expensive option.

Anyways I don’t feel too strongly about this either way, I just think the argument here that the iPhone is a luxury device doesn’t absolve them. Other arguments I’m more sympathetic to is all.

szvsw··on Apple must ensure interoperability of iPhone with rivals, says EU
> It's a luxury product.

25-30% of smartphone owners have an iPhone. Something like 1.3b active iPhone users. That seems decidedly non-luxury. Neither here nor there but that’s part of your argument that seems pretty flawed (ie: it’s a luxury product so we shouldn’t have to worry about the vagaries of “anti/pro-consumer behavior, competition etc”). Sure most of Apple’s other products are in the more to much more expensive end of their categories, but the iPhone’s market penetration definitely makes the luxury categorization wrong IMO, even if it is expensive.

Anyways, your entire argument effectively boils down to “this problem does not affect me, therefore it is not a problem; I don’t care if there is a monopoly on some product or service or lock-in or anything because I’m fine with it, I like the service and have no plans on leaving anyways” - which is a position plenty of people take, and also one plenty of people disagree with, so I won’t try to argue either way with you here since you seem committed. It does seem short-sighted and generally anti-consumer to me (or like you are just evidence that their various lock-in strategies work and are good for their business - which you self-identified as being correct in re: intentionally buying into walled-garden).

Anyways, that’s not to say I totally disagree - I think most of your points are generally true/or something I could at least see being true even if I don’t feel convinced (ie potentially ending up in a sea of third party crap, creating a potentially undue burden on Apple engineering etc… tho even those I find questionable and not self-evident).

← PreviousPage 4 of 10Next →