4,368 karma · joined December 2, 2012
Reach me at keno@juliacomputing.com
Quite possibly yeah. Jameson had to do quite a bit of work to get the backend to a point where it would reliably work on the IR we generate. Now, julia does have a pretty wide coverage of LLVM IR, so it's probably not terrible, but as always the test cases used in development determine what works and everything else probably requires some work (actually very similar to the regular LLVM backends - whenever we port to a new architecture we first end up having to fix about a dozen bugs to do previously unexercised code paths).
| Is there a wider interest in resurrecting this for wider use?
We don't currently have an active use case. Jameson has been maintaining it on low flame and merging pull requests/doing some cleanup here and there, but since we're not actively using it, there is of course a limit to that. There has been some activity on the repository recently though, so you're not the only one interested.
Basically what you end up doing is discretizing spacetime on some grid, maybe e.g. 100x100x100x100 (not much larger than that last I checked or the problem becomes intractable even on very large computers) and then sample field configurations monte-carlo style (the way to do it is actually really clever - it turns out that there is a mathematical trick that can take you from the description of your theory right to a distribution from which you can draw your field configurations). The number of fields is decently large (some discretization introduce extra auxiliary fields for every physical one) and in general you need to consider the entire field configuration to compute an observable. Then do that a few billion times (to bring down statistical error) at a couple different lattice spacings (to be able to extrapolate to the continuum limit) and you quickly end up with a very large computations.
There is a number of clever ways to reduce the amount of computation required, but in general it's a very different problem.
Hope that makes some sense. I had looked into this in detail about two years ago, but looking back, I feel like I only know every other word in my write-up ;).
I'm perfectly happy to acknowledge that there have been conflicts in the community. However, I maintain that the overwhelming majority of interactions in the community are immensely positive. Please do also consider that there may be parts of the story that you are not seeing in the public issues. I do hope you will consider giving the community a second chance. You have quite clearly identified some of the major technical challenges we face and we'd love to hear any ideas you have to address them.
I do think HN is not the right forum to link to individual comments and call out people for their behavior. Perfectly happy to discuss the technical issues though. If you would would still like to discuss, or at least bring such instances to my attention, please do feel free to send me an email.
> People who suggest refactoring large sections of Julia's expansive base library (1600 symbols? a problem that causes parts of the slow startup time and memory usage) into other libraries (which would improve batteries included, etc. features) are summarily ignored / shouted down / (predicated on my previous paragraph) banned.
There are several recent examples where people have suggested such things, e.g. They were neither ignored nor shouted down nor banned. https://github.com/JuliaLang/julia/issues/16357 https://github.com/JuliaLang/julia/pull/16070
> Important improvements to the language go in circles on JULEP tagged issues namely any large improvement that would require multiple people and lots of dev work seems to gain little traction unless a core dev just takes the time to do it; rather than a timeline to reach consensus and then design a plan for implementing it.
I don't understand what exactly the criticism is. At first I thought it was about, too much dicussion, but then it seemed to be about too little dev time, please do clarify.
> There was a push to increase test coverage and then a few months later commits plummeted the code coverage numbers and no one seemed to care.
I'm not really sure which instance you're referring to, but last time there was a major drop in coverage, it was investigated, found to be a bug in the instrumentation (which was fixed). I don't really think it's fair to say that people don't care. There's somewhat of a tooling problem here, since we can't run the coverage tests on travis, so we don't get them integrated in the GitHub UI, but people do look at them and add tests as appropriate.
> Sections of important code are uncommented, undocumented, and untested on purpose.
Yes, there are undocumented and uncommented sections in the code base, but I wouldn't say that they are so on purpose. I do admit to having added such hacks in the past (and not documented them because they were gonna go away soon after), because I needed them in outside packages, but people don't let me do that anymore ;).
> Unpredictable un-typed exceptions, namespaces/modules being hard to use for non-trivial designs, interfaces without any sort of type enhancement.
These are fair points, which I'm sure you've seen the issues about. The problem is the availability of developer time, not some sort of unwillingness to fix problems. Putting together the road map of what goes into each release and prioritizing are very hard, because there's just so much that could be worked on.
> setup a transparent community policy and actually follow it;
This is being discussed (as part of larger discussions around community governance) and as I mentioned will likely be a topic at JuliaCon.
I can only think of a couple of instance where I've heard anything like this.
The first and most prominent is Dan Luu. I'm very sad about losing him from the community. I've met him a couple of times in person, I still read his blog and I very much respect him. I don't think any of us have the full story of what happened there, but I sincerely hope that time may be able to smooth things over.
The second was in an HN comment that I just went back to find, and discovered that is was you. I'm sorry if you had a bad experience with the community. I know my perspective is biased, but I've rarely interacted with a community that's as passionate, helpful and friendly as the julia community (LLVM is up there as well).
As for documenting the process, I agree there could be some improvement there. We do have a contributors guide (CONTRIBUTING.md in the main repository). As for policies for feature inclusion, we've considered having more formal code owners for parts of the code base (right now there's mostly de facto code owners for various parts), but haven't found it necessary so far. I definitely expect some of this to be discussed at JuliaCon in June.
- Startup performance/memory usage
Yes, we are definitely very acutely aware of these. Julia is not currently optimized for frequently run short scripts. That's the price on pays for having to bring up the entire runtime system (initializing the compiler, RNG, external libraries etc). The good news is that there will be a solution to this soon, which is to statically compile your julia program. The area where this really comes up for most people using is package load times. We're very actively working on making that faster. - Syntax
A little subjective, so not sure how much I can say here. I can say that I'm not a huge fan of our multi-line comment syntax. It's not entirely clear what a better syntax would be though (the original issue on this had some suggestions, but some of them were worse). - One-based indexing
I think there has been plenty said on this topic, though interestingly this is one of the only times I've seen the argument made in a way that I actually agree with. That said, I do think there is an easy way to deal with this though. For packages that needs arrays of indices, it would be quite easy to define an `IndexArray` type that does the translation automatically. - String Formatting
Yep, you're right, it's a mess. It'll have to be cleaned up. - Unsafe C Interface
There's two projects (Clang.jl and Cxx.jl) which can help with this. The former automatically generates ccall definitions for you, the latter just parses the header and generates the call directly. - Slowing down in development
I'm really not sure where that impression comes from. Perhaps it is that we're adding fewer features, but rather working on cleaning up existing features. Also, I personally at least have been doing a lot of work outside
of base (particularly on the debugger). Not sure. Would love to know.