HNHacker News
TopNewBestAskShowJobs

DNF2

325 karma · joined August 9, 2018

submissionscomments
DNF2··on Julia: Faster than Fortran, cleaner than Numpy
But this is what most languages do, except they have _singular_ dispatch. Every OOP language has different behaviour for different input types, except it only works for the _first_ input argument. Julia simply extends this to every argument.

How would you feel about having to write `sqrt_int32(x)`, `sqrt_int64(x)`, `sqrt_float64(x)`, `sqrt_rational_complex_float16(x)`, etc, etc, etc, etc, instead of just `sqrt`, and let dispatch or overloading take care of the different implementations?

DNF2··on Julia: Faster than Fortran, cleaner than Numpy
In mathematically oriented code, long variable names make code less readable, not more. Mathematical expressions become long and difficult to parse, and the variables often have no intrinsic meaning, so trying to make up 'meaningful names' is pointless.
DNF2··on Julia 1.6 addresses latency issues
It think that the discussions on HN tend to be quite combative, and members of the Julia community (including myself) can get a bit worked up over negative claims that we find overly sweeping. That could lead to some dismissiveness.

I do get frustrated by characterizations of the interests of the Julia community that are very broad, and are also in evidence here. E.g. somewhere upthread one person claimed Julians are only interested in Jupyter Notebook workflows, which is a totally alien statement to me. Jupyter doesn't seem to have a very large mindshare as far as I can tell, I've never even tried it myself.

Static compilation and analysis comes up regularly on Discourse, but among the scientist part of the community, there is naturally less emphasis on this. Please don't extrapolate too far based on some isolated interactions.

DNF2··on Julia 1.6 addresses latency issues
I wonder where you got the impression that latency and precompilation performance are not valid concerns. This has been the _main_ focus area for the devs for a long time. It's pretty much all anyone has been talking about for over a year, and serious improvements have been made.

Here's a blog post that goes into some detail about the ongoing efforts to improve compiler latency: https://julialang.org/blog/2020/08/invalidations/

DNF2··on Ask HN: Charisma (faked) trumps (real) technical skills?
Perhaps. It is possible that a community just cannot handle an extremely high level of social and communication skills. It becomes too much for them.

The same thing can happen to politicians too; some of them are just so gifted, well-informed and good with people that everyone starts disliking them.

It's an interesting phenomenon.

DNF2··on MLJ.jl: A Julia package for composable machine learning
The two-language problem is not concerned with calling out to or wrapping pre-existing libraries in a second language. It means having to write parts of your code in a second language, due to, for example, performance reasons.
DNF2··on MLJ.jl: A Julia package for composable machine learning
So then that 'weakness' in python is not an issue to you personally, because you fluently drop down to a fast language anytime you need to, with no particular loss in productivity?

Keep in mind though, that this could be a hurdle to those who are less multilingual. So even though it's not a weakness to everyone, it is to many.

DNF2··on MLJ.jl: A Julia package for composable machine learning
I beg your pardon? Knowing that Python is slow is a strength of Python? Then, if Python suddenly became fast, it would be a negative?

Did I misunderstand something?

Presumably, all fast languages are at a disadvantage, then.

DNF2··on Julia 1.6 Highlights
Not just floats, and I'm not sure they have to be that small. All sorts of structs containing bitstypes/value types can be stack allocated. In fact, even some structs with pointers to heap-allocated memory can be stack-allocated (such as array views.)

I don't know about linked lists, though.

DNF2··on Julia 1.6 Highlights
This is a Windows issue, I'm pretty sure. My solution is easy, install Julia under WSL. In fact, after moving from MacOS to Windows, this is my goto solution: install as much as I can under WSL.

Problem disappeared.

DNF2··on Julia 1.6 Highlights
This depends on what you mean by '(un)optimized code'. Because there's a difference between unoptimized and naive code.

'Unoptimized' code should still observe most of the performance tips in the manual (such as avoiding globals and type instability), while 'naive' code frequently does not. With some experience, you never write naive code, even for quick prototypes.

In those cases, Julia should outperform other dynamic lanuages significantly, and approach static languages in most cases.

Proper optimization means going in and removing allocations, ensuring that operations vectorize (simd), tailoring data structures for performance, adding parallelism etc. In the latter case Julia should virtually _always_ match static languages closely, otherwise it merits investigation.

DNF2··on Julia 1.6 Highlights
I don't have a strong opinion on this, one way or the other. It's less verbose (which I like), and more familiar to those used to dynamic languages like python, matlab, etc. But this isn't my decision, I'm ok with either.
DNF2··on Julia 1.6 Highlights
I have no particular opinion on the teaching materials, I just use the manual and the discussion fora, so I don't know. But if a third party offers teaching materials, it's not so strange if it resides on their third party website.

As for performance, I'm not really talking about 'optimization'. Your implementation may simply have used some pattern that should be avoided, such as global variables, type instabilities, abstract types in structs, or some inappropriate data structures. If it's a microbencmhark, then there are some things to keep in mind.

These are not really optimizations, but basic performance principles. I cannot know that you are unaware of them, but your statement that 'there's not a lot to optimize' make me suspect that this could be the case. The unusual thing about Julia is that it's both dynamic and compiled, so that code that would simply not compile in static languages instead ends up slow.

DNF2··on Julia 1.6 Highlights
I simply do not understand how some people are able to form so strong opinions in such a short time, and spew out disdain and negativity on the most flimsy basis. It's a matter of temperament, I guess.

Julia performance should be on par with Go, if it's slower, read the performance tips in the manual. As for teaching material on 3rd party websites, I don't know what you mean. The Julia manual is available from the julialang.org website.

As for re-writing DifferentialEquations, that is extremely strongly tied to the multiple dispatch paradigm, re-writing it would be hard. What you can get is wrappers like diffeqpy and diffeqr, which call out to Julia.

DNF2··on Julia 1.6 Highlights
This is just a disagreement over basic design: should variable declarations be explicit or not. It is a choice, and something that reasonable people can disagree on.

Framing this as a case of irrational and illogical behaviour is unnecessary and unreasonable in my opinion. A lot of serious thought and debate went into the resolution. There is no need to disrespect and badmouth people because they have different priorities than you.

DNF2··on Julia 1.6: what has changed since Julia 1.0?
When do you run into this kind of problem? Is it possible that you are over-typing your function signatures? The generally recommended style is to write generic code with loose type restrictions, or even none at all.
DNF2··on Julia 1.6: what has changed since Julia 1.0?
No, this is incorrect, you are confusing backwards and forwards compatibility. Running your new code on old Julia versions could break immediately, just like in every programming language.

Backwards and forwards compatibility have very different horizons.

Running old code on new Julia versions should not break until the next major version. Packages, on the other hand, are different, and could break your code sooner, but that's the same in any language.

DNF2··on Julia 1.6: what has changed since Julia 1.0?
Well, a lot of resources have been put into reducing compilation times, and large improvements have been achieved. It's not just perennially 'around the corner', the improvements are tangible and happening right now.
DNF2··on Julia 1.6: what has changed since Julia 1.0?
But this is really the worst conceivable use case for Julia. Why are you interested in making this change, when Julia offers no advantage in this scenario?
DNF2··on Julia 1.6: what has changed since Julia 1.0?
If it were 20 seconds for every plot, this would be a major problem for me, as I tend to make lots of plots. But it's only the first plot where this is an issue. Surely you're not in 'the zone' that soon?

Seems to me like people are making a mountain out of a molehill.

DNF2··on Julia 1.6: what has changed since Julia 1.0?
I cannot fathom how ttfp is important. It's time to first plot, not to every plot. After it's down to ~10 seconds, why on earth does anyone care?
DNF2··on Julia 1.6: what has changed since Julia 1.0?
No one interested in interfaces? My impression is that interfaces is very commonly discussed, and is one of the most anticipated features in the language, though it may not come until v2.0.
DNF2··on DSP Performance Comparison Numpy vs. Cython vs. Numba vs. Pythran vs. Julia
I submitted an issue that gives a further >4x speedup with just simple straightforward code (no simd, fastmath or threads).

It's important to make everything a view, and to remember the dots in the right places, otherwise this is quite straightforward.

I believe you can get significant _further_ speedups with better simd vectorization and threading. I made this as simple as possible, to be similar to the python/cython code.

DNF2··on DSP Performance Comparison Numpy vs. Cython vs. Numba vs. Pythran vs. Julia
Yes it does. There are still issues with benchmarking in global scope, though, which is why it is better to use BenchmarkTools.jl
DNF2··on DSP Performance Comparison Numpy vs. Cython vs. Numba vs. Pythran vs. Julia
I had the impression that the inputs were one 2D and one 3D array. Are they both complex? Also it seems like they did not have the same sizes, as well as dimensionalities.
DNF2··on DSP Performance Comparison Numpy vs. Cython vs. Numba vs. Pythran vs. Julia
I commented on this, right next to your question: "Well, view could speed up code, but it also had some overhead, so creating lots of views came with a cost that would sometimes outweigh the benefit."

So yes, there was indeed perf improvements previously, they were simply not as lightweight as they are now.

DNF2··on DSP Performance Comparison Numpy vs. Cython vs. Numba vs. Pythran vs. Julia
Well, view could speed up code, but it also had some overhead, so creating lots of views came with a cost that would sometimes outweigh the benefit.

But, yeah, version info would be interesting.

DNF2··on DSP Performance Comparison Numpy vs. Cython vs. Numba vs. Pythran vs. Julia
That's good. I still cannot see the repository, though. I may be looking in the wrong place.

It's no problem being new to a language, I just found the conclusions as bit too 'conclusive', so to speak, in that case.

Views should definitely not slow this down, so there may be something off. Could you perhaps share some dummy input arrays? Can I generate them with rand, perhaps, if you can tell me the sizes?

DNF2··on DSP Performance Comparison Numpy vs. Cython vs. Numba vs. Pythran vs. Julia
That Julia benchmark is really quite unfortunate:

1. It treats all arrays as row-major, while Julia arrays are column major. That is pretty unfair. 2. It creates lots of unnecessary copies, as slices in Julia does not return views (but you can easily make them do that.) 3. It creates an output array with an abstract element type, `zeros(Complex, pols, L)`, which can seriously hurt performance. 4. Benchmarking should preferably be done with BenchmarkTools.jl, though I'm not sure it that makes a big difference here.

It's ok to be a novice user of a new language, but when comparing and publishing benchmarks like this, I really think the author should solicit some feedback to avoid obvious rookie mistakes. I found no way of offering any feedback on that blog, however.

DNF2··on Julia adoption keeps climbing
All right! I got the impression you were contrasting that particular post with the rest of this discussion, but apparently not. Still slightly confused here. Oh well, carry on.
← PreviousPage 5 of 7Next →