245 karma · joined September 6, 2020
I want to see a DSL that can be used to describe models elegantly and then export them either to a shared object or to something that can be run with a runtime (in this case IREE). Things like ONNX and TorchScript promised this but I've had little luck getting these to work well enough to trust them in large scale production deployments.
I understand that PyTorch is an awesome tool for researchers, but it doesn't necessarily fit into a prod environment.
I do think you make a great point, which is that the risks and rewards need to be considered. The problem is that analyzing the cost is also a pretty complex problem and there are very real arguments to be made that going green could end up being very costly.
FWIW, I actually came into the climate science debate on your side and have been won over by paying attention to a broader set of sources. One of the things I really can't stand about the climate change disaster is that even if you accept all the claims from the IPCC, it pales in comparison, in my opinion, to other issues that seem a bit simpler to me. The collapsing insect populations, for instance, can have a pretty direct impact on human health and happiness. The climate discussion seems takes all the attention even though no one has made a credible argument as to why it is more dangerous than other problems we face.
edit: I'll add that scientific modelling is _very_ difficult in general, even for much simpler systems than what we are talking about here. These models have tons of simplifying assumptions baked into them and they can be made to spit out just about anything, depending on what values for certain parameters are chosen. The tools aren't particularly trustworthy.
edit: I'll add a bit more. I do think climate change is happening and I do think that humans might have a big part in why it is happening. What I don't find compelling is why this is necessarily an end-of-the-world scenario. The typical argument is: "there has already been warming and there will be further warming. Then there will be runaway effects, then the warming will become extreme and life will become very difficult for most people in most places." Well I do agree with the first part, but it is _really_ hard to predict what happens afterwards, i.e. are there going to be runaway effects. There are reasonable arguments that there won't be. The arguments that there will be tend to focus on a few scientific facts and ignore the complexity of the system at large (i.e. they ignore negative feedback mechanisms and focus only on positive feedback).
Anyone who says "the science is settled, the laws of physics cannot be violated" fails to realize what physics is even about. The vast majority of victories that physics has had since Isaac Newton are based on incredibly simplified models that strip out all of the complexity from a system. Basically everything interesting is treated as linear (as in f(x) ~ x) and a lot of the other stuff is just thrown into constants. After a few decades of working with that, both in theory and with experiments, you might get comfortable enough to add some small corrections to your theories. In this way you step closer and closer to "truth" but we are still very far off, even with our best physics, from fully understanding anything of even moderate complexity. This is not to say that we don't get incredibly useful results from physics (just look at our technology!), but it does mean that we need to constrain statements about how much we really understand and where our "laws" really are applicable. Take for instance the equation for kinetic energy, k=.5 * mv^2. Hugely useful, but dead wrong if you try to apply it to things moving at relativistic speeds.
The problem of climate science, from my perspective, is that we can't strip it down like we do for simple systems like a ball moving in space. Climate is inherently very complex; if you try to ignore how oceans interact with vegetation, how vegetation interacts with clouds, how clouds and rain are connected, etc, then your models can be interesting and might reveal something, but they can't be used to predict what will happen with any degree of confidence because they are too far removed from the real thing. And if you don't strip something out of the system when doing your modeling, well then good luck: you'll never understand anything at all because the thing is too complex to be used to calculate anything of interest.
edit: added some words.
edit2: more words.
edit: grammar
> I actually got a lot done
In an article that seems to be promoting the idea of relaxing and taking a break once in a while, the author couldn't resist sneaking in nod to the typical fetishistic obsession for constant productivity that is endemic in this crowd. It seems like we're told that it is possible to relax AND be productive. Is it so unreasonable to disconnect from work every once in a while without worrying about being "productive?"
> Metrics are useful
No they aren't, not for this. There is something seriously wrong with someone if they need a $300 piece of equipment (+ a few hundred more $ for the phone to connect to it) in order to understand something so fundamental about their bodies. We aren't talking a diagnostic imaging to find a suspected tumor here. We're talking about being tired. This is absurd.
EDIT: Small typos
EDIT: I'll add that last fall I was biking around there with my wife and we found some most righteous porcini mushrooms growing on the side of the road. It really is nice out there.
I should also add that nearer Paris is Forêt domaniale de Meudon, which is in the direction of Versailles. Nice forested area. I can't speak to the walking/hiking but I assume it is not bad (I've only been out that way on bike).
For me, thankfully, there was an easy solution: I moved to one of the towns just outside of the city. Now I have ample access to lots of trees, birds, flowers, etc. If I get on my bike, which I do frequently, I can be in some really beautiful regional parks within 40 minutes or so, and the ride there ain't too bad either.
Apparently this can be changed by requesting that FF only install updates with explicit consent[1]. I'd think the best way would be to install anything that's available locally when the browser starts without forcing a restart.
[1] https://superuser.com/questions/1451210/how-can-i-make-firef...
Know what you are doing and make sure the people you are talking about are knowledgeable.
EDIT: I am not disputing your claim, BTW. But I think it is just like the fact that all US doctors took a few semesters of organic chemistry doesn't meant that they can tell you anything useful about the subject.
I do think cars have their place, but I don't understand how people have come to believe that cars in, say, Manhattan or the center of Paris, make a lot of sense.
It is true that per engineer, successful SV firms do make a lot of money. But it is _really_ hard for me to imagine this class of people as "ignored and underpaid." FB, Google, etc are publicly traded companies, not co-ops and, as such, are under no obligation to pay any more than they do. Don't like it the terms? Don't sign the contract. They make it sound like they make you sign a contract with a gun at your head.
> A Facebook software engineer’s average starting salary is about $150,000 a year...But outside of San Francisco, it’s probably equivalent to around $50,000 a year.
This is just stupid. A _starting_ salary of 150k? Even if the equivalence they draw to 50k/yr elsewhere is correct this is a pretty good place to start. I made 95k starting out in SV and had plenty of money to burn.
This article is not rigorous and just bad.
The science funding climate, at least in the US (I don't really know what goes on elsewhere) is in a really sad state. Projects with a low probability of success are really hard to get funded but that is precisely what you need to come up with really novel things. Curiously, science funding bodies don't seem to think of things in a probabilistic way.
The second issue is the same idea, but applied to industry. There is an activation energy that you need to surpass in order to make any business profitable. If we are talking about building a business around a brand new and unproven (in a business sense) technology like mRNA, the barrier is _huge_. You need entities with very deep pockets to accept year after year of losses before you can start getting a viable business producing cutting-edge products.
In both cases I think intelligent use of government funds could be a big help. I am not big on socialism, but I have long held that this is one of the most important things a government can and should provide: subsidies for risky research and business endeavors, not to mention large-scale infrastructure projects.
> Yet, it's good to write software in such a way, in which it is easy to replace one part with another when a better (for the time being) alternative shows up
I think few would dispute that, but it isn't exactly an easy thing to do. You could spend 100% of your time adding abstractions to even moderately complex pieces of software to make sure that every last functional component could easily be replaced with some other implementation but then you wouldn't actually get anything done since all your time would go to creating and implementing _just the abstractions_. This is pretty impractical so you really have to choose your battles: which components are likely to be replaced in the future and is it worth worrying about it now? I find the vast majority of the time it is really hard to accurately predict future constraints and requirements so I tend to default to very simple choices that make sense given what I already know. When something really feels like it will need work in the future, I might add a relevant abstraction. Otherwise I am basically just wasting time.
In short, I view KISS, in the context of software engineering, as an admission that you can't really predict the future so it isn't worth dwelling too much on preparing yourself for it.
EDIT: typo
In this case it is not difficult to get at the cited document from which the author's claims are drawn. For those interested, it is available here:
https://www.cmadocs.org/Portals/CMA/files/webinars/COVID-19%...
These data don't necessarily mean anything without an understanding of the context, but they are definitely useful and shouldn't be ignored out of hand, even if the author is using them to support a conclusion different than your own. In other words: accept the data for what it is worth and then use it to form/strengthen a reasonable conclusion.
EDIT: Punctuation
Generally, I think the best strategy is to be open to what people might have to say if one is also diligent about checking sources.
EDIT: Missing words.
Two things that might be tweaked:
* The search didn't behave in an ergonomic way: I typed a query ("graph neural networks") and great relevant stuff came up immediately in the dropdown. When I hit enter, however, I got an error that read "Invalid search: Check your spelling, enter a DOI, or another paper identifier or." I would have expected my action to take me to a search results page that listed what I saw in the dropdown (which I regard as a preview of the top hits) so that I could peruse the selection carefully.
* I wanted to load a paper to take a look at it and it took me a while to realize that I could click the "Yes" above "Open Access" to download it. Since one of the big use cases for a site like this is the eventual consumption of these papers, I suggest making a "read/download paper" call to action more explicit.
I don't think that's quite relevant, though. Rust doesn't have a concept of nulls like like Java, Javascript, Python, etc do in which just about any variable might contain a value or null. Rust certainly doesn't have that.
Other languages give you more flexibility in how to share data between processes.
The fundamental claim that threads running pure Python are limited by the GIL still stands. Others do point out that you can get around this in C and some of the standard libraries in Python (like hashlib) do this for you. That goes a long way to helping the issue, of course, but as yet others point out, it is weird to support Python in this context by saying "Yes Python can use threads effectively; all you have to do is use C."