Julia 1.0
julialang.org
julialang.org
Nim's home page¹ shows a piece of sample code right at the top. Perl6's page² has a few tabs quickly showing some patterns it's good at. Golang³ has a dynamic interpreter prepopulated with a Hello World.
Julia's home page shows a nice feature list and links to docs to deep dive but it doesn't do a good job of selling it.
Issue filed: https://github.com/JuliaLang/www.julialang.org/issues/115.
Lisp languages do take a long time to get used to.
The first impression does leave me thinking that using Julia for different programming domains like distributed internet-facing servers or web services is not something it was built for.
No. Julia is a general purpose language that has so far been mainly focused on scientific and mathematical programming.
It's design is probably least friendly to the real-time programming domain (GC based) but it can apparently be used there as well:
I see an extremely bright future ahead for Julia!
https://github.com/JuliaRobotics/RigidBodySim.jl/blob/34ac43...
Notice that this function is purposefully sleeping the differential equation solver in order to slow it down to the exact amount to get the simulation back to real-time.
So like Python, you have SciPy Pandas etc. but don't have to leave Python when you need to do a bunch of text processing or whatever.
At the same time, calling into other languages (eg C) is super straightforward (disclaimer: last time I checked).
And of course, I should really assist to solve this problem myself. But, I thought to encourage other language experts.
http://docs.juliadiffeq.org/latest/features/io.html#Tabular-...
f_2dlinear = (du,u,p,t) -> du.=1.01u
prob = ODEProblem(f_2dlinear,rand(2,2),(0.0,1.0))
sol1 =solve(prob,Euler();dt=1//2^(4))
using DataFrames
df = DataFrame(sol1)
# Result 17×5 DataFrames.DataFrame │ Row │ timestamp │ value 1 │ value 2 │ value 3 │ value 4 │ ├─────┼───────────┼──────────┼──────────┼──────────┼──────────┤ │ 1 │ 0.0 │ 0.110435 │ 0.569561 │ 0.918336 │ 0.508044 │ │ 2 │ 0.0625 │ 0.117406 │ 0.605515 │ 0.976306 │ 0.540114 │ ...
Exception handling in julia is poor, which reminds me of how exceptions are (not/poorly) handled in R. Code can trap exceptions, but not directly by type as you _would_ expect. Instead, the user is left to check the type of the exception in the catch block. Aside for creating verbose blocks of boilerplate at every catch, it's very error prone.
Very few packages do it right, and like in R, exceptions either blow up in your face or they simply fail silently as the exception is handled incorrectly upstream by being too broad.
Errors, warnings and notices are also often written as if the only use-case scenario is the user watching the output interactively. Like with R, it's possible but quite cumbersome to consistently fetch the output of a julia program and be certain that "stdout" contains only what >you< printed. As I use julia also a general-purpose language to replace python, I feel that julia a bit too biased toward interactive usage at times.
That being said, I do love multiple dispatch, and julia overall has one of the most pragmatic implementations I've come across over time, which also makes me forget that I don't really like 1-based array indexes.
Exception handling is indeed somewhat of a wart with the core system not having changed much since very early on. I think there's still some serious design work to do and the core people take it seriously. On the other hand, I suspect that numerical codes don't want to use exceptions for much other than "panic and escape to an extreme outer scope". So a lot of the scientific users are probably content for the time being.
On the other hand, I'm wondering if this will help a little with the dependency hell that's caused me to drift away from Julia over the last year or so.
At first I was fairly excited about Julia, and greatly preferred it over R or Python for numerical work, library resources aside. It was fast and I liked the language design itself.
Over the last year or two, though, I've had recurring problems with trying to install and use packages, and them failing at some point due to some dependency not working. Sometimes the target package itself won't install, but most of the time it's some lower-level package.
It's more frustrating than not having packages available, because it creates some sense that packages are available, when they're not. At first I looked at it as some idiosyncratic case with one package, but it's happened over and over and over again.
Basically in this time I've given up on Julia, because there's such a huge discrepancy between what it looks like on the surface and what it looks like in practice, once you get out of certain limited-use cases, or when you're not coding everything yourself from the base language. (Related concerns about misleading speed claims have been raised, although my personal experience in that regard has been mixed, because my experience overall has been pretty good in some critical performance cases, but there have also been some wildly unpredictable exceptions that act as bottlenecks... but it's still much better than R or Python).
When I've tried to figure out what's going on with dependency hell, usually it's because some package that was previously working with a earlier version is no longer working. So maybe stabilizing the language will help that?
If you can, try and find a few hours to help pitch in and solve the package problems, even if it's just updating docs or updating deprecated-but-useful packages.
The current major issue, as it stands, is that it's very easily for a malicious bit of code to sneak into a heavily used JS package and have oversized effects - this happened very recently with a very popular linting-support package.
The other issue is general posting of malicious packages under mistyped names, or takeover of existing packages with malicious updates by new owners.
At the same time, nobody wants to have NPM (the org) manually vet every upload ever made. So, there's that.
Many JS packages are extremely dep heavy, overwhelmingly for minor features (checking something is an array, promise-fying FS, etc) which makes it very easy to infiltrate packages and very hard to vet a package entirely.
Finally, npm (the program) runs into a fair bit of caching woes and it's own dumb bugs which feel like they shouldn't slip into production nowadays. Oh, and sometimes npm (the website) goes down.
The answer for JS, unfortunately, is probably segmentation - as better managed and more secure package repos come up, likely with their own package managers, npm will probably have to up their game. That, I am sure, will bring a whole fresh set of issues.
With the new binary builder, binaries should be another note, so long as you download an official Julia binary, or built Julia from source in the same way those binaries are made (eg, build OpenBLAS with gfortran 7).
But more generally things are maturing.
And 1.0 will help too, since things won't be chasing a moving target.
If your not in any hurry, I'ld give it 6 months, of people who don't mind a bit of packages breaking (e.g. people like me) using it.
That will be plenty of time for everything to shake out. More than you might expect has actually already been shaken out in the last few weeks in the package ecosystem. Hitting 1.0 should give some package maintainers the drive to get it done.
That said, I am optimistic Julia will have a good solution at some point: contextual dispatch, a la Cassette.jl, enables just the sought of interventions you want for error handling. I'm not quite sure what the result will look like, but I imagine you will see some experimentation in this direction in the near future.
I initially defended the choice, but I now agree that 1-based indexing now seems like a poor choice since Julia has become something more than the original mission of a better MATLAB or Octave. It’s a, admittedly, minor tragedy of Julia’s success.
>In order to read a csv in that doesn't have a header and for only certain columns you need to pass params header=None and usecols=[3,6] for the 4th and 7th columns:
https://stackoverflow.com/questions/29287224/pandas-read-in-...
Just reading that hurts me.
Again, 0-based indexing exists to fit a purpose: http://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EW...
In my opinion, reading `a = b[1:n-1]` hurts much more than reading `a = b[:n]`.
No, that's their ordinal position, and how 8 billion non-programmers would refer them as in any everyday setting.
That's also how programmers would refer to them if it wasn't for a historical accident.
What's more, that's also how programmers refer to them when they talk between then and not to the machine ("check the 3rd column" not "check the column at the index of 2").
I have taught Python quite a bit, and I have gotten good at explaining 0 based indexing and slicing based on it. When I switched to Julia there was nothing to explain. And my code has about as many +/- 1s as before...
These spoken language conventions developed before there was an established name for “zero” or even a concept that “nothing” could be a number per se.
For similar reasons, we have no zero cards in our decks, no zero faces on a dice, no zero hour on our clocks, no zero year in our calendar, no zeroth floors in our buildings, East Asian babies are born with age one year, etc.
It’s only by another set of historical accidents that we have a proper 0 in our method of writing down whole numbers. Thankfully that one was of obvious enough benefit that it became widely adopted.
I’m curious as to why this is a problem outside numerical computing. From my perspective, this is consistent with a long history of mathematics dealing with matrices that predates electronic computers.
0-based arrays are popular because C decided to deviate from what had previously been standard in math and in Fortran.
Is there a reason other than aesthetic preference and habit that makes 0-based indexes better for computing in non-numerical contexts?
I realize both indexing standards are arbitrary and boil down to “that’s the way grandpa did it,” however 1-based indexing grandpa is way older and more entrenched outside computing circles.
EDIT: I suppose with Julia it’s not that important, as other commenters have pointed out that you can choose arbitrary indexes.
An unexpected takeaway: Michael Chastain's time-traveling debugging framework, which he had in 1995, and which still reads like sci-fi from the future[1].
Alas, this quote will probably stay relevant for a while:
>[We keep] finding programming constructs, ideas and approaches we call part of “modern” programming if we attempt them at all, sitting abandoned in 45-year-old demo code for dead languages.
Then the second parts quotes the creator of BCPL, revealing – a-HA! – what we already knew, that the array is a pointer to the first element and the index is the offset from the pointer, and that's why it's zero.
(And after that it veers of into complaining about how the papers documenting this history cost too much money, so they didn't actually read them.)
1-based indexing has the advantage of always knowing the length of the array since the index of the last element is this value. You also get "proper" counting numbers as you iterate.
I've used both in various languages. Perl lets you define the array index start to be whatever number you wish.
People argue that zero-based is incidental, and that 1-based is the right way because of it's long history in mathematics notation. I would argue that 1-based is incidental, and that zero-based is better most of the time for modern math and computer architectures.
This points to another example where 0-based should be preferred. When doing modulo arithmetic, 0..N-1 mod N gives 0..N-1, but 1..N mod N puts the zero at the end. I also cringe at languages where -1 mod N does not equal N-1.
Especially since it would have been very easy to 'do it like Ada' and allow any start index by default (I have use Lua and it's really annoying to use an extension language with 1-index when the base language is 0-indexed)
For a pre-baked solution: https://github.com/JuliaArrays/OffsetArrays.jl
Instead of complaining in the abstract, check it out, you'll be impressed. https://julialang.org/blog/2017/04/offset-arrays
(not referring specifically to you here!)
Until then Fortran, Lisp, Algol, Pascal, Basic never had any big issue with indexes.
Programming languages should not be configurable in that kind of way.
You can get around some of the pain of haning error types in catch statements if you are comfortable paying the price for a locally defined function:
result = try Something() catch err h!(e)=throw(e) h!(e::MethodError) = SomethingElse() h!(err) end
This pattern works well if an error case should return a default value and all others should throw.
For example, the fact that an array of unions such as Array{Union{Missing,T}} is represented in memory as a much more efficient union of arrays is a perfect example of where a clever compiler can make the logical thing to do also the efficient thing to do!
Once you make that distinction, then whether you write it as a Cython module exposed in Python or you can use native language features to do it in Julia, nobody cares. It’s encapsulated away from people who use numerical libraries, as it should be.
I also spend time developing these “runtime overhead avoidant” backend numerical libraries, and I would say I’ve seen no significant reason to prefer Julia over Cython.
Don’t getme wrong, Julia is great, just not offering anything fundamentally different. And since there’s already a critical mass of people with engineering and optimization experience in the Cython & Python extension module stack, I’d expect that community to continue dominating Julia just by attrition alone.
http://www.stochasticlifestyle.com/why-numba-and-cython-are-...
I don’t think Julia’s generic code is the end game for fast code though. Efforts in Python toward code gen include projects like loopy, which targets accelerators, are still developing but make it easy to decouple kernels from loop domains, target specific devices and optimizations etc.
Modern numba can also do a lot more for huge scale projects than what the article suggests.
Julia docs also seem very smugly proud of multiple dispatch and autogenerating implementations for multiple types or signatures.
But Cython fused types allow the exact same polymorphic multiple dispatch. Here’s an example pedagogical project illustrating that point.
< https://github.com/spearsem/buffersort >
This pattern is quite easy and offers a lot of generic strategies in Cython, especially if you just want a bunch of overload options in a pure C backend with a thin entrypoint to Python.
With some persistence Python-Cython-Numba can get the work done, but to me it has never felt like a consistent whole, usually a tagged on hodge podge. Numba is cherry picky on what it will and will not optimize, I totally understand why it is so. It has always been a 'will it or wont it" with Numba.
I think the potential that Julia offers that Cython and Numba doesnt is the option of a programmable syntax for the low level manipulation. Cython has improved, but for the longest time it was always a forced decision between operating at a the higher level of array/sub-array manipulation in Python or down to tedious index manipulation in Cython with nothing in between. Not to mention fighting the Cython compiler and Python's C API to get the 'yellows' out.
Cython fused types are required to be known at the package's compile time, and that's exactly the point that makes it less composible. Not requiring this is exactly the advantage of Julia which is demonstrated.
The article doesn't say Numba cannot do huge scale projects. It just talks about the difference in the compilation strategies. If you read that and think that it means Numba isn't suited for huge scale projects, then that's your interpretation of how Numba works, not an opinion directly stated by the article.
The article gives an easy way for you to demonstrate it wrong. You can show 10 lines of code where you send dual numbers and uncertainty objects through SciPy's ODE solvers to generate solutions with gradients and error bars. If Cython can do this, please show it.
And when I want Julia’s promise of fast loops, I use Numba. If all the effort gone into Julia had instead been spent on fixing remaining warts in Python workflow for science, we wouldn’t even havee this conversation.
For package developers it is a lot eaiser to use Julia.
Saying one should focus more on Python and we would not have these problems is missing the point. Enormous resources by countless companies has been poured in to solve the performance problems of Python.
It is almost impossible to do due to the language design of Python. You cannot fix it without breaking the language.
Julia in contrast required minimum effort and resources to get fast. It is almost a toy project compare to Python. It is all down to clever language design which allowed them to use a rather dumb and simple compiler while letting LLVM do most of the heavy lifting.
Every advance of Python is going to require 10x the effort of advancing Julia. It will just be a question of time before Julia catches up. Python has a huge lead so that will still probably take years but it will happen.
Python’s design leaves something to be desired performance-wise for those coming from JVM or native languages, but it’s a trade off, not an obvious win (for Julia), and the problem goes away as programmers get wise to performance strategies in Python.
An example of the value of generic code is that forward mode AD is extremely easy, and almost always just works on whatever code you run it on.
Then, once that's done, multiple dispatch (and possible macros for a DSL) allows for a much cleaner user interface than Python offers for numeric code.
I have a lot more experience with R than Python, but seeing more of the scientist/mathematician/researchers side of things, I have to strongly disagree with the view that they should write slow code and contact a CS guy to write fast code in another language when they need it. Do you honestly think that's practical for grad student's projects? Recently, one of my friends wrote a simulation in R. Most of the work was done by hcubature -- a package written in C -- integrating a function written in R. Could just have easily been written in Python. That function was slow, and the simulation ran for days. Before an error caused it to crash, losing days of compute time. I -- a statistics grad student -- helped him rewrite it in Julia, and it finished in 2 hours.
That C/C++ code will still run slowly if they have to call your R/Python code is a problem. They also can't apply things like AD easily. A common solution, used by Stan for example, is to create a whole new modeling language and have users interface through that. Learning a new language -- albeit relatively simple/domain specific -- which they then cannot debug interactively, is another pain point. All this can be avoided by simply using Julia.
What I see on Github is as professional as it can get. Issues, discussions, triage, review, CI-tests for example.
Maybe you started too early, before Julia was settled? And/or were too over-enthusiastic to begin with? I think Julia had to grow, find the 'correct' solution with e.g. NA/Missing/Nullable. Break things b/c it didn't work out as expected. Postpone things, debugger (maybe?), for more important areas or because base was not stable yet.
Two years ago in a project I hoped that people would switch immediately from R to Julia. But in retrospect it was good they didn't. Julia was not ready for them and too much ecosystem stuff missing/unclear still. (This said, Julia would in principle have been much much better suited for that project).
I get the issue with scientists writing poor code, but Numba has largely solved this problem, by packing an LLVM JIT into a decorator which can be applied to any numerical code to get same speed ups as Julia, except no language switch required.
Citing slow code in the wild with a fast rewrite is a hilariously poor anecdote performance wise. I’ve rewritten Fortran code into Python and gotten speed ups. Regardless of the language, garbage in, garbage out.
Stan is an example where the modeling is “just” a DSL implemented as C++ templates. Does that make that a good choice?
This only works (easily) as long as you don't have user-defined types
> If all the effort gone into Julia had instead been spent on fixing remaining warts in Python workflow for science, we wouldn’t even havee this conversation.
Python is too dynamic, you cannot just fix remaining warts. From Julia documentation I know that Julia language has been designed for speed and e.g. some dynamic possibilities have been omitted in order to be able to generate fast code. For a general idea about Julia speed see e.g. this 7 hours old excellent Juliacon video: https://www.youtube.com/watch?v=XWIZ_dCO6X8
Python certainly works. But already for syntax alone, if you have written Julia, it's hard to - in my case - go back to R.
https://numba.pydata.org/numba-doc/dev/user/jitclass.html
No one intends to fix Python but it’s straightforward to do things like Numba: use a decorator to read out the AST for a function, reimplement it however you like and pass back the compiled function, and document the semantics.
Python is rather a mess. Code written in Python can't be sped up without pain/cost, and apparently it will never support concurrency natively. It also suffers from the bane of weakly typed languages, errors at run time instead of compile time.
I think the sweet spot for a language with most of Python's benefits that fixes many of its glaring warts is enormous.
It’s not weakly types either: you can’t add a list to a string. It’s dynamically typed.
In fact I think it's a much better thing to have a sort of division where a general scientist can put together a simulation/analysis pipeline in Python or R, but if they want to implement new algorithms they'll be sufficiently out of their league that they'll need help from someone who has actually spent time learning how to code properly and efficiently, to do testing and version control etc.
In fact it's very similar to how experimental science often works: you have the general scientist who just knows how to do basic measurements with some instrument, and you have the instrument scientist who has to help if the general scientist wants to use some novel technique, who keeps the equipment tidy and working and logs everything in the big lab journal.
... and it's been wildly successful. Some of the most prolific and talented contributors to Julia and its ecosystem are scientists by day and brilliant programmers of all kinds—crazy meta programmers, compiler hackers, generational GC writers, etc.—by night (as they say). During the keynote last night before tagging 1.0, we went through the history of major features in Julia's development history, it became a running theme that so many of these contributions have come from people who are physicists, chemists, geologists, biologists, etc. So I'd say that Julia is strong evidence that breaking down the separation between scientific users and developers of scientific libraries is a good idea.
Can you elaborate on how Julia represents arrays of unions, or point to some documentation? I'm working on something where an automatic efficient representation would be useful, and I'd like to learn from other's experience.
In terms of the Julia changes that were needed to support this, there were two key PRs:
https://github.com/JuliaLang/julia/pull/20593 https://github.com/JuliaLang/julia/pull/22441
https://youtu.be/1jN5wKvN-Uk?t=1h3m
It was fun to do this with everyone at JuliaCon and online, and thought it was worthwhile to share here.
For example, I have downloaded julia-1.0.0. I try to follow this tutorial here, linked in this post by someone: http://juliadb.org/latest/manual/tutorial.html
Then I do this and get an error:
julia> using JuliaDB [ Info: Precompiling JuliaDB [a93385a2-3734-596a-9a66-3cfbb77141e6] ERROR: LoadError: UndefVarError: start not defined Stacktrace:
Every time. Even the screenshot of Julia code that julialang.org used to have was not runnable per admission of core devs.
What am I doing wrong? How are you able to run large Julia programs successfully?
Edit:
Let's try tutorial at https://www.analyticsvidhya.com/blog/2017/10/comprehensive-t.... First command: Pkg.add("IJulia"): command fails to install dependency Conda.
Same for the tutorial at http://ucidatascienceinitiative.github.io/IntroToJulia/Html/...
Sigh. Give up
You should have tried 0.4.2, 0.6.3, 1.0.1. Motto: Don't start too fast and be slow giving up ;-)
That would help a lot of people get started.
More informative wikipedia page: https://en.wikipedia.org/wiki/R_(programming_language)#CRAN
That being said, R was GNU S once upon a time, so they didn't really build a new language, rather an open source clone of a popular proprietary tool. (In case it wasn't clear I both love R and am madly excited that Julia has finally hit 1.0.0).
The Julia package ecosystem is much more anarchic, like npm. You basically just have to have a public git repository with certain files in place, and a cursory review from the managers of the package metadata.
There are tradeoffs. I'm honestly not sure which one is the right way. I really appreciate how much I can trust that a CRAN package works, but there's a reason so many R devs are using devtools to do an end-run around it.
Which package declared support for 1.0.0 and didn’t compile?
One thing I found is that types in Julia are first-class values [0]. You can put them in variables, pass them around, inspect them, even produce new ones in runtime. Opens up all kinds of metaprogramming opportunities. Very Lispy! (Well, Julia is Lispy). It's also interesting that types are optional, yet they're significant for the optimising compiler. Like, if you do type the arguments of a function, the optimising compiler will have less work to do.
It also looks like much of the compiler pipeline is exposed to the user [1]. You have access to the parser, can tweak the AST. Macros also there of course.
[0] https://docs.julialang.org/en/stable/manual/types/ [1] https://docs.julialang.org/en/stable/manual/metaprogramming/
Giving types for function arguments doesn't actually have any effect on performance: the compiler specializes on concrete runtime argument types anyway, so completely untyped code is just as fast as fully type annotated code—since the types are known when the code is compiled. On the other hand, giving type information is essential for performance involving memory locations, i.e. field types and the element types of collections.
I hope Julia doesn't rely on inferior libraries just out of copyleft phobia. I would much rather use FFTW than FFTPACK or whatever other alternative they have in mind. FFTW is really best in class.
I'm okay with them making FFTW optional, but please make it opt-out, not opt-in. People should be getting the best software by default. Copyleft isn't going to hurt anyone but people who are trying to hide source code, and scientific computing needs all of the visible source code we can get.
FFTW is available in a package under the MIT license [1]. Also its author is a top contributor to Julia ;).
> Note that FFTW is licensed under GPLv2 or higher (see its license file), but the bindings to the library in this package, FFTW.jl, are licensed under MIT. This means that code using the FFTW library via the FFTW.jl bindings is subject to FFTW's licensing terms.
If you have an idea on how to make that clearer, we would be happy to review a PR to the FFTW.jl repository.
Julia's Base is for the language, not for every little detail so this is all handled by the package ecosystem.
Their type system is pretty interesting, and allows for some really cool abilities to parameterize things using types. I'd like to have seen more work done on, effectively, strong typedefs (or whatever $lang wants to call them). However that sort of thing is fairly uncommon so it's hard to hold it against them too much.
The biggest issue, and one they seem unwilling to really address, is that actually using the type system to do anything cool requires you to rely entirely on documentation which may or may not exist (or be up-to-date).
Each type has an entirely implicit interface which must be implemented. There is no syntax to mark which methods must be present for a type. No marker for the method definitions, no block where they must be named, or anything like that. You can't even assume you'll find all the interface methods defined in a single file because they can appear literally anywhere.
Whoever wrote the type has in mind some interface, a minimal set of methods, that must be present for any derived type. There are only two possible ways to determine this. The first is to look to the documentation. Even for the basic types defined by Julia this documentation doesn't seem to exist for all types. I don't have high hopes for most libraries to provide, and keep up to date, this documentation either. This concern gets even greater when considering the focus is largely on scientific computing.
Without up-to-date documentation, the only option is to manually review every file in a library and keep track of the methods defined for the type you're interested in. With multiple dispatch, you can't even get away with just checking the first parameter either. Then you need to look at the definitions for those methods to narrow your list down to the minimal set required. This is not an easy task.
This issue has been brought up before and discussed, but nobody seemed very interested in it. This is a fairly major issue in my view, as it cripples the otherwise very interesting type system. As it stands, it seems to be a fairly complex solution to the issue of getting good compiled code out of a dynamic language. It could be so much more.
The main reason this has not been implemented as a language feature is that people are worried about settling on a design that would be impossible to make fast and concise. It is certainly on the designers' radar, and was discussed specifically at JuliaCon 2017 in Jeff Bezanson's talk.
There are some people who plan to attempt a trait system as a package on Julia 1.0. Perhaps this will be successful and we won't need language changes! Stay tuned.
As an aside, I wouldn't take the lack of action as lack of interest. People are interested, but it wasn't prioritized yet. It will get effort and attention!
First, this extends deep into the core of Julia, and I don't see how a traits package would be involved with that.
Second, and this dovetails with the first issue, this needs to be something that people actually use as a default action. Part of that means ensuring the built-in types make use of this.
Related to all of this, I worry it's too late to really make a meaningful change here. The culture and existing packages are already set without it. Adding the feature as a requirement isn't going to happen anytime soon unless people are willing to continue to break things post 1.0. And it needs to be a requirement or it won't get used except by people that will already provide documentation.
The lack of interest I mentioned mainly came from some github issues which either received little attention, had creators that had a very "maybe it would be nice if" attitude, and responses that were questioning the benefit compared to the cost of implementing the syntax changes.
No, Tim Holy described over dinner how all that was necessary to complete the existing traits packages was method deletion, and that's in v1.0.
But as an external package it still has the issues I described as not being part of the defaults of the culture. Without that it just becomes a nice-to-have that only people serious about writing good quality, usable code are going to use. And these are the people most likely to have good documentation in any case.
As a comparison, consider that the C++ standards body has been working on, debating and serially rejecting the various "C++ concepts" proposals for years now.
If you want to get fancier, yeah, it gets hard. But something as simple as a syntax block that lists the interface methods would be enough for now.
The fancier stuff is already projected so far out to at least 2.0 that I don't understand why a simple working solution wouldn't be desired for now. It would also simplify the work of changing code later to work with 2.0.
I really don't understand how this issue wasn't addressed a long time ago, and why it wasn't a blocker for 1.0.
`methodswith` tells you all methods that been defined on a type. Since it returns an array of methods you can do some more introspection on that, if you're so inclined.
I've also just written a few test cases using the interface I'd like, then worked on methods until my tests pass (which can be just not throwing MethodErrors).
I would also bet (but not too much) that we eventually see major progress in removing the GIL. I really don’t think it’ll be around forever!
One starts with quick and dirty solution, makes it work on a small dataset and then struggle to make it utilize at least 4 cores to cut running time with more realist datasets. Surelly I can code numerical calculations in C++, but then the code cannot be maintained by python-only guy. So I hope that Julia or anything else with better parallel support replaces Python for scientific calculations when scaling quick and dirty solutions is straightforward.
[1] http://scipy-cookbook.readthedocs.io/items/ParallelProgrammi...
I think this is so badly needed to get an easy route to pipeline paralellism, which is simply everywhere in today's data analysis "pipelines".
https://docs.julialang.org/en/latest/base/multi-threading/ https://docs.julialang.org/en/latest/manual/parallel-computi...
It's listed as experimental since in a 1.x released it's planned to be changed to work on top of the task interface.
> [...] it may change for future Julia versions, as it is intended to make it possible to run up to N Tasks on M Process, aka M:N Threading
M:N threading is (I think) the same as the "multiplexing" I mentioned. Have seen it called "M:N multiplexing" before.
(At the very end of https://docs.julialang.org/en/latest/manual/parallel-computi...)
* Julia has had tasks/co-routines basically forever and uses them for all blocking operations so that no explicit non-blocking I/O or callbacks are required.
* It also supports multithreading using the @threads macro.
* However, it does not yet map tasks to threads, but that is very close to ready: https://github.com/JuliaLang/julia/pull/22631.
We expect this work to be finished in a near-future 1.x release and then Julia will support M:N multiplexing.
It's multi threading model is mostly multi process. Mostly because people write code to run on clusters on machines, not just single ones.
co-routines, channels, etc will be nice, and I expect not too far away.
I am ambivalent over whether it has multiplexing or not. When I need parallelism I fall back to OS threads if available or processes. One or the other is usually always a first class citizen. Whats not that common, are real lightweight coroutines.... I want my laptop to be able to simulate the transport layer over a sizeable portion of the internet, while I watch youtube.
Very few pieces of software combine simplicity and power. I'm very excited.
It'd be great if Julia devs could help deploying Julia Pkgs on Nix, now that Julia is getting ready for prime time and Nix is steadily gaining more and more users: https://github.com/NixOS/nixpkgs/issues/20649
This makes a poor first day impression for new users who expect Plots.jl, IPython.jl, Juno.jl or other prominent packages in 1.0. The package maintainers are scrambling to catch up. I recommend using 0.7 for a couple weeks until the community catches up unless you are proficient enough to contribute PRs and help out.
(edit) To be clear, 0.7 was released at the same time as 1.0.
And perhaps more importantly, what is missing?
I'll give you one quick example of the design. Images are just arrays of "Colors". The upside is that you can write generic code that handles arrays of Colors of any kind, meaning you don't have to worry about iterating over the color axis. You also don't have to worry about whether your 100x100x3 array is three images (at different times or locations) or one image with three RGB channels. All of this comes at little to no cost in terms of speed.
Check the docs here: http://juliaimages.github.io/latest/
But fair warning, it still needs some fixes for 0.7/1.0. We're on it!
With respect to your question about what’s missing: Coming from Python, I think one might be surprised by how much less one feels the need to rely on packages in Julia. The language has some very powerful abstractions that really makes one rely less on packages than you’d expect.
Julia has really inhereted a lot of great lessons from Lisp and that’s made the language an absolute treat for ‘rolling your own’. Meanwhile, some lessons from more modern languages have also made Julia much more effective at sharing your abstractions with others much easier and effective than any Lisp I’ve ever seen.
Julia v0.6 was already very functional/usable release but having 1.0 allow me to write code and have confidence that it will continue working in the same, solid foundation in the years ahead.
I started my coding career over 25 years ago and I had done much work in C, Java, JavaScript, and PHP. Julia is the best language that I have worked with so far. Fast, dynamic, and highly productive!
I'll be moving to 1.0 release when my dependent packages are upgraded to support 1.0.
But for classical scientific computing, of the sort typically done by real engineers wearing coke-bottle glasses, using MATLAB and expensive toolboxes, it can't be beat. It's really easy to work with matrices and make your code super-fast. And it steals all the good parts of MATLAB syntax but with a good, modern, sensibly designed programming language.
Also, DifferentialEquations.jl is undeniably the best available piece of software for solving differential equations, in terms of both performance and ease-of-use. It feels like using alien technology or something.
Julia is not only good for its performance, but also its multiple dispatch, its type system and more. Because of those features we have https://github.com/JuliaGPU/CUDAnative.jl, more elegant package interface, like https://github.com/JuliaOpt/JuMP.jl and there is more.
I still believe there is no silver bullet, if you want something Python then you get another Python. But think about this: is the full Python's dynamic feature really what we need in works like simulation, HPC, etc.? Probably no. I think Julia is kinda of balance for the related field (like for scientific computing). Just like in the old days, people start using FORTRAN, MATLAB, Lisp as the most advanced tools at that time. We start using Julia now.
The point is mostly to (a) show the difference between fast languages compiled to efficient assembly and (b) represent code someone new to a language may bang out to get something done (while avoiding performance pitfalls) and avoid the benchmark game.
That said, I agree. I try (poorly) not to advertise speed, because people coming from languages like R will rarely fail to write type unstable code that is slow, observe JIT compilations that make Julia a little laggy, and then come away disappointed.
Things like the type system and meta-programming shown off in your examples are amazing, and also not something you can reproduce in other languages by adding binary dependencies.
?
I'm not an PL expert, so I would like to ref a previous discuss in discourse, which is more detailed
https://discourse.julialang.org/t/julia-motivation-why-weren...
I was just trying to say that if you want to develop something with reasonable performance, elegant interface in a short time. Probably under this situation, Julia is more suitable.
Perhaps because the Julia homepage distracts them :-)
"Julia is fast! Julia was designed from the beginning for high performance."
As a more general point though, I work in an academic environment and I see tons of languages whizzing by. I routinely see C, C++, Python, R, Matlab, csh, bash, and the list just goes on and on. I've even seen someone write a command line tool in PHP that had no justifiable reason for being written in PHP. But it works, so whatever. If it was written in Julia, maybe it would be uncommon, but also slightly more refreshing.
So, really, why not Julia?
$ time ipython -c '1+1' Out[1]: 2
real 0m0.493s user 0m0.401s sys 0m0.063s
$ time julia -e '1+1'
real 0m0.222s user 0m0.115s sys 0m0.101s
Yes, python itself is still much faster. I compare against ipython just because that used to be my go-to interactive shell for technical computing.
The real question is how long does `using Plots` take in v1.0?
I think I started using Julia around 2013 or so after getting fed up with the slower speed of octave, so I'm guessing that things should be a fair bit smoother from this point on if they've officially tagged 1.0.
0-based indexes are commonplace in programming circles because of the C language, however 1-based indexes are the earlier standard set by Fortran.
EDIT: If Julia becomes popular outside data and numerics circles, it will have pulled off nothing short of a miracle in getting people to adopt 1-based indexes. This feud is older than vi vs. emacs. :-)
One-based indexing has a good, long history.
It will for sure hurt adoption.
Reminds me of the complaints my friends had about Linux when I first showed them a fantastic window manager.
"This will never get anywhere. It has no Start button."
If I were in a team that refused a language just because it is 1 based indexing, I would really worry about the abilities of the team.
1 based indexes were just fine.
It's also not hard to get used to. No more OB1 errors.
If you are calculating an index the domain and range of the index function are very often 0-based. So you have to subtract and add one to convert from and to 1-based. It's just messy.
That makes no sense. Neither 1-based indexing or 0-based indexing will save you from OB1 errors.
As a matter of fact, if you make more OB1 errors in a 0-based indexing language, it's probably because your brain is wired to think 1-based.
The problem: there are legions of programmers whose brain is wired to think 0-based and are guaranteed to suffer through a lot more OB1 errors if they try to adopt Julia.
0 based indexing is not because of C's influence. It's because that's how computers work. ASM is 0-based indexing. The first memory cell on a computer doesn't start at address 1.
When I do sigmas in math, they go: i=0,i<n and very rarely i=1, i<=n unless the problem really is made simpler by the weird 1-based indices.
- Julia is by far my favorite language. (I've also written significant code in Java, C++, and Matlab and small projects in Python, Mathematica, R.)
- Julia is my favorite because it is super expressive but also fast. You don't have to make (big) compromises. There's a great blog post called "Why We Created Julia" with the punchline "we are greedy." [1] 6 and a half years later, it holds up well.
- In Julia, nothing hurts. There are so many little quality of life improvements that add up to more than just quality of life. Some are small, like multiple assignment (x, y = lst[1], lst[2]). Others are more conceptual, like well-supported first-class functions (that are also fast). Another example: you're not forced to write code in an arcane style or with special libraries to get speed. Your normal for-loop or vectorized code or functional code will all compile to something efficient.
- Because Julia is fast and expressive and extensible, in Julia everything can be Julia and not a mashup of other languages. I've been doing some work in Python recently, and it's painful to have Python lists, numpy arrays, Pandas series, and so forth. Converting between types isn't that hard, but it's real mental (and textual) overhead which just doesn't have to be dealt with in Julia.
- Yes, Julia has 1-based indexing by default. There are packages for custom array indices (including 0-based, symmetric around 0, pick your favorite) which are, surprise, super performant and easy to use. It seems uncontroversial to me that for some cases 1-based indexing is a more natural mental model and for some cases 0-based is more natural. When it matters a lot, you can pick your indexing. When it doesn't matter much, which is most of the time, it doesn't matter. Julia catches a shocking amount of flak for this...if the worst thing about a language is that it sometimes makes you add or subtract 1, you must really like that language :)
- The Julia package ecosystem is young and evolving. It has some standouts such as DiffEq (differential equations) and JuMP (optimization modeling language) which are, to my knowledge, best-in-class in any language. I'd say the modal experience is more like DataFrames: already super functional and productive, not yet as full-featured as the <popular language>-equivalent, and slowly evolving towards something better than the popular language equivalent. E.g. DataFrames is just a wrapper around Julia lists which makes it much lighter weight / easier to understand / easy to interop with than Pandas.
- There are some growing pains around a young-ish language which, until today, hadn't reached its first stable version. Presumably those will taper off now that we're at 1.0, but it'd be a lie to say there aren't any.
- My first open source contributions, modest as they are, are all in Julia. Pre-Julia I never knew how to get started, but Julia makes it easy to transition between user and developer.
This is exactly what I like about Julia. Even though getting started in Julia was tough (esp the older versions), once you get off the ground Python starts to seem like a very hard language, in that Python requires you to use these special libraries to run fast code. In Julia, the most obvious way to do something (e.g., for loops) is perfectly fine.
The main thing Julia lacks, for me, is an equivalent to Pandas. The DataFrames library lacks many very useful features of Pandas. But I am sure someone will tackle that.
The #1 requirement I have is the ability to make binaries for some program. You can compile a C program and get a binary. There's no practical equivalent for Julia at the moment and I think this limits its production potential.
Not had chance to how nicely PackageCompiler.jl works but am hopeful :)
One of the reasons I used Julia in academia was slurm manager ;)
then in computer, (assembly) you will request an element by saying for example base_offset+(1elem_len). that would make it logical to use 0 as an offset, because then you can use 'nelem_len' as a generic number to incrememnt the offset by to select array elements.
thus for a computer and how it functions, 0-based array would make more sense, and any other thing, would just be some overlay over how a computer works just to make it more human readable....
computers dont care for what is first idex, it will always equate to base+n*elem_len to have to find the actual location in memory...
if people have been discussing that for decades and what is better, it's just another example of people not understanding what is an opinion,and what is objectivley true... none is better, your compiler is taking care of business, really it is, and no longer like it was 1999... they have been patched many times!
And what about for a human and how it functions? Are humans here to make computers' lives easier or vice versa?
Humans think from 1...N inclusive and this is the source of a litany of bugs when users first learn a language.
And what you described in asm is just one implementation. In fact, the array documentation says Julia doesn't guarantee tight packing so it doesn't even apply. The next element could be anywhere, and in fact wont be in the case of heterogeneous arrays. Asm is also not straightforward. For instance zeroing a register isn't done with the mov instruction but xor
"computers dont care for what is first idex...none is better, your compiler is taking care of business"
I think you're assuming everyone thinks the way you do. I assure you that's not the case. There are legions of people (and not just programmers, math folks do it too) who don't "think from 1 to N"
As for "a computer" - unless your actually writing code for memory allocation - you are (should be) writing to an abstract machine anyway (trivially, if you want to sum numbers in a dense 10^6 by 10^6 matrix - wouldn't it be nice to utilise 1024 cores if they are available? Unless you have side effects happening, do you really care about the order of additions? And if the numbers are 256 bit integers - do you want to care if they're stored big or little endian, in 8 or 16 bit words? In two's complement?).
It seems that the first step in getting Jupyter to know about a new version of Julia is to do Pkg.add("IJulia") in Julia. Except that that doesn't work; it seems that now you're supposed to use some special pkg mode in the Julia REPL.
So, I hit ] to enter pkg mode and type "add IJulia", which seems to be the appropriate thing. It churns a bit, tries to build something called "Conda" (which is apparently the dependency-management bit of Anaconda, the Python distribution thing), and gives me an error message that starts like this: "ERROR: LoadError: ArgumentError: isdefined: too few arguments (expected 2)" followed by a stack trace whose first and last entries are "top-level scope at none:0", which doesn't exactly help to nail down where the problem is.
Related operations like "build Conda" and "build IJulia" give similarly unhelpful error messages (some of them enjoining me to do things like Pkg.build("Conda") that so far as I can tell don't actually work at all).
Do I just need to wait for release 1.0.1, or is it likely that I've done (or left undone) some unfortunate thing, that I could fix and make everything work?
I thought it might be interesting to try the Juno IDE, but met with a similar lack of success: first of all it told me I needed to do Pkg.add("Atom"); when I had done (not that but) the approximately equivalent ]add Atom, starting Juno yielded only a cascade of error messages (no method matching eval(::Module, ::Expr); failed to precompile Media; failed to precompile Juno; failed to precompile Atom).
Presumably, again, the answer is to wait a little for things to settle down. It feels as if it might have been better to get all the ducks in a row before declaring version 1.0, though...
Of course, more time would have been nice, but you can always find a reason for a delay. At some point you have to rip off the bandaid.
It will take awhile for it to fully utilize v1.0. Remember, it just came out so packages will need to catch up. The debugger for example needs to be updated, along with the plotting packages and such. But once everyone updates it should give a very similar experience.
The best documentation I can find is this article, but it seems a bit spartan on how-to: https://medium.com/@sdanisch/compiling-julia-binaries-ddd6d4...
So while you might find the language itself pleasant, you'll probably find a lack of libraries.
(personally I'm a fan of strong static type systems (e.g. haskell), and I think it is a shame the designers didn't go down that path. But hey, they did all the work and not me, so who am I to complain. Kudos to the 1.0 milestone!)
In julia, you can write a custom type, and immediately have access to all of the builtin libraries.
For example, I wrote a drop-in replacement for floating points, and immediately had complex numbers, matrix math, gaussian elimination, fourier transform, etc... And could rapidly compare the numerical performance of that with fp.
For a more exotic example, I wrote a galois field GF256 type and was immediately able to reed-solomon encoding and decoding using the matrix multiplication and matrix solving libraries.
For an even more exotic example, I wrote a "lexical GF256" type (basically passing strings) and had the system generate optimized C code for reed-solomon decoding, using the builtin julia matrix solver without having to manually do matrix solving for every possible (n,k) reed-solomon system.
It was also relatively easy to write a verilog generator (~3 days of work), so you could pass bit-arrays representing wires, run you unit and property tests on the binary representation, then redispatch the function passing lexical types and get verilog code out the other end, then transpile the verilog to C using verilator, dynamically load the verilog into the julia runtime, and then unit and property tests on the transpiled verilog.
I'm sure it's possible to do this in haskell, but I imagine it would be harder.
To get good performance with a static type system, you need generics. Which means longer compile times, as the libraries you use must be compiled as part of your project so that the code specialized for the types you're calling the library with is generated. You can't have piles of native code lying around just waiting to be used (like in the C world, for example).
But this is similar to how Julia does it at runtime, with the multi-methods being JIT-compiled for the types you're calling them with. So either way you pay the cost somehow (and yes, there are ways to reduce that overhead, of course).
I've been working on updating a wide variety of engineering software apis/wrappers and a large informatics library for .7/1.0, but I'd love to hear any suggestions of must-need technical packages that users want from MATLAB, Python, or R.
> square(x) = x * x
> @code_typed square(2.0)
CodeInfo(
1 1 ─ %1 = Base.mul_float(%%x, %%x)::Float64
└── return %1
) => Float64
> @code_typed square(2)
CodeInfo(
1 1 ─ %1 = Base.mul_int(%%x, %%x)::Int64
└── return %1
) => Int64
julia> square(x) = x^2
square (generic function with 1 method)
julia> @code_native square(1)
.text
Filename: REPL[3]
pushq %rbp
movq %rsp, %rbp
Source line: 1
imulq %rdi, %rdi
movq %rdi, %rax
popq %rbp
retq
julia> @code_native square(1.0)
.text
Filename: REPL[3]
pushq %rbp
movq %rsp, %rbp
Source line: 1
mulsd %xmm0, %xmm0
popq %rbp
retqI strongly prefer the dependency management system to python's efforts.
I like the strong arithmetic precision management facilities.
Dynamic programming is interesting, but I'm not sure that its great for collaboration.
Sounds like 1.0 hasn't propagated everywhere yet :-)
https://gist.github.com/ontouchstart/5222903bf4a2d040e6eee6a...
… they forgot to compare Julia to JavaScript — the most popular programming language in the world.
Or is that because authors don't want Julia to be used by anyone?
¯\_(ツ)_/¯
[I'm joking, ofc. But I still find it weird how almost every modern language is mentioned, but JavaScript…]
I know it is not necessary, I was just curious, as thr language seems to have a lot of metaprogramming options, and Femto-Lisp being part of the compiler.
Is TCO on the roadmap, or is the "mess it does with stack traces" too much?
https://github.com/chrisvoncsefalvay/learn-julia-the-hard-wa...
No, it's the same indication, twice. Once you've decided on 1-based, you almost definitely want to include both endpoints. Else you end up saying things like a[1:n+1] to indicate "the whole array please", which is annoying.