HNHacker News
TopNewBestAskShowJobs

KenoFischer

4,368 karma · joined December 2, 2012

Julia Core Developer Co-Founder at Julia Computing

Reach me at keno@juliacomputing.com

submissionscomments
KenoFischer··on Julia on Google TPU: Shakespeare RNN
You are correct that there is a social aspect to making julia packages work well. That's part of the reason so many julia packages are organized under various GitHub organizations to make sure that these kinds of discussions have a place to take place. However, I don't really see that as a negative thing. It gets package developers to talk to each other and delineate the abstractions for their packages more clearly. And in the end, I don't see it all that different from package development in any other language. Your users will always do things that you didn't intend and then you have to decide whether their use case is in scope for your project or not and act accordingly.

Aside: this is not TensorFlow, but XLA, which are two very different things. It's also possible to try this kind of thing and generate a TF graph, but TF is a much less nice compilation target.

KenoFischer··on Julia on Google TPU: Shakespeare RNN
So to me the biggest semantic question are the following:

  - Does what you have to write to run on X follow the semantics of the language?
  - Can you use data structures/code defined in libraries that don't know about your thing?
In the case of julia on TPU here, the answer is yes! (surprisingly perhaps, and getting this to work is pretty hard). In particular, you get a lot of julia's language features: Multiple dispatch, control flow, etc. It's a bit of a subset of the full language (e.g. no mutation at the moment), but everything that's supported is just standard julia and we're working on growing that subset more and more.

That approach is very different from something like TensorFlow where you're essentially metaprogramming an expression graph. Numba probably counts for python (yeah, you have to put an annotation on things, but if the python people really wanted, they could probably import Numba into core cpython and make it work more smoothly). Of course in python, you have the additional complication that most of the core implementation is not in python itself, so even if you satisfy my two criteria above for the core language, you're still gonna have to rewrite the whole standard library.

KenoFischer··on Julia on Google TPU: Shakespeare RNN
There's very little TensorFlow here. We're using it as a glorified grpc client basically to ship XLA to the TPU. The actual mechanism is described in https://arxiv.org/abs/1810.09868. The model definition itself uses the layers from Flux, but there's a couple assumptions that Flux makes that don't hold for the TPU so we don't get to use everything from Flux (e.g. the training loop and the optimizers). Luckily everything is julia, so we can get TPU-compatiable versions in just a few lines of code. Unifying the abstractions is an active work in progress, but this shows off that Julia runs on TPUs at all (an on freely available, public infrastructure at that), which is pretty cool, because right now the only other TPU frontend that can say the same is TensorFlow (which had a bit of a head start ;).
KenoFischer··on Julia on Google TPU: Shakespeare RNN
The notebook is about 400kB. It also downloads a 4MB dataset and julia, tensorflow and dependencies need about 400MB, but that gets stored in the ephemeral Colab VM, not on drive if you copy it there.
KenoFischer··on Julia on Google TPU: Shakespeare RNN
Sure, the point is

  1. To use a small dataset that can be fed from the relatively small Colab VM (so people can play with it themselves)
  2. To use a well known model (so people can focus on the implementation parts of it)
There's a resnet example in the previous example in the series (though the dataset is too large to feed from Colab).

Also, this is very early stage software, so things are a bit more verbose than they should be :).

KenoFischer··on Growing a Compiler: Getting to Machine Learning from a General Purpose Compiler
Well, I do think there are some very real shortcomings that current general purpose compilers have when applied to machine learning (aggrevated by the fact that lots of machine learning code is written in languages with poor compiler support), so lots people looked at that, wrote small optimizers and got good performance. But then it turned out that researchers wanted to do more and more things in their ML models and these small optimizers are turning into full-blown compilers, with not a lot of thought about whether that is truly the correct thing to do.
KenoFischer··on Growing a Compiler: Getting to Machine Learning from a General Purpose Compiler
This is a talk we gave at the "Compilers for Machine Learning" workshop at CGO this past Sunday. Obviously the slides are missing some of the context of the presentation, but the core argument is that one shouldn't build compilers for machine learning, but rather build general purpose compilers that are flexible and extensible. From there, you can easily get all the same benefits as dedicated machine learning compilers without building a huge monolith. The rest of the slides highlight some of the work done in the Julia community to work in this direction.
KenoFischer··on Deep learning may need a new programming language
I always think these discussions miss the fact that static type systems only give you these benefits if whatever property you're trying to prove is encoded in your type system. E.g. people keep telling me that static type systems will prevent shape mismatch errors at runtime, but then go ahead and give their variables a static type of "Tensor" (without shape information), because encoding that shape information in the static type system is a bit of a pain and nobody really wants to write that (Not always, but it's a surprisingly common pattern).

Meanwhile, we have absolutely no problem statically checking Julia code (and in fact we do when generating code for TPU). It's a bit of a non-standard thing to do and we're working on improving the tooling for this, but putting Julia on the same level as python here ignores a significant aspect of the design of Julia.

As for collaborations among thousands of engineers, it is true that we don't really have Julia projects that large yet (other than Julia itself) and I'm sure we'll undoubtedly find things to improve in the language to support collaboration at that scale, but i really would encourage you to look at things like the new package manager or the documentation system. A significant amount of work has gone into the usability thereof and I think at that point they warrant criticism more detailed than an off hand dismissal.

KenoFischer··on Why Swift for TensorFlow?
That is basically the core insight behind julia. The really performance sensitive parts of your application are already static just by the nature of that code, so we can extract that static information to make it really fast. We can also make use of the same static information for static error messages or static compilation and get the best of both world (dynamic during development, static when you're done), but the tooling for that is a bit less developed at the moment.
KenoFischer··on Differentiating SSA-Form Programs in Julia (2018) [pdf]
We have these numbers now and the results are quite encouraging (basically on par with tuned tensorflow for the same model, while retaining significant flexibility). At the moment, we're working with Google on stability and waiting for a new public release of the tpu software stack to enable multicore support for non-TF frontends.
KenoFischer··on Differentiating SSA-Form Programs in Julia (2018) [pdf]
Hi folks, this is part of of larger effort in the Julia community to do a ground up re-think of infrastructure for machine learning. You can find an overview of the entire effort in our recent blog post [1]. Happy to answer questions.

[1] https://julialang.org/blog/2018/12/ml-language-compiler

KenoFischer··on Running TensorFlow at Petascale and Beyond
Traditionally the HPC community means double precision when they talk about FLops, which consumer GPUs are not super great at (somewhere around 400GFlop or the GTX1080 Ti). The V100 is better at 7 or so TFlops, but also an order of magnitude more expensive. The smaller of the two machines in the article (Cori) is a ~20 PF (Float64) scale machine. However, even then it is quite challenging to get anything close to peak performance out of these systems (a bit easier if you're doing DL will is more amenable to hand-optimized vendor libraries).

Of course for DL, people routinely use reduced precision as you have alluded to. There, we're looking at exascale at the moment. The second machine mentioned in the article (Summit) has ~3EF of Float16 performance (of which their application reached ~1EF). For comparison (and assuming about 50% MXU utilization - which is about the maximum I've seen), you'd need about 200 TPUv2 pods (at $384/hour/pod for the general public) to reach the same scale on this problem. Now Google does have that scale, but it's certainly non-trival.

Half-precision petascale is fairly routine these days. You can get that on the cloud for < $100/hr and I do routinely spin up such systems for ML training. However, petascale fp32/fp64 and exascale (b)f16 as discussed in the article are still fairly rare and usually preceded by months of planning to make sure things go right and the computer power is used usefully.

Disclaimer: not involved in the work, but working closely with the folks who are and I've used both the mentioned systems myself.

KenoFischer··on Julia v1.1 Release Notes
Worth pointing out that after the 1.0 release we've moved to a time-based release schedule, so this is mostly just accumulated niceties from the past three months. Lots of bigger things bring worked on separately .
KenoFischer··on Julia Language Co-Creators Win James H. Wilkinson Prize for Numerical Software
The difference is run-time (semantically) vs compile time. If you had overloading at runtime in C++, you wouldn't need virtual functions or any of the thousands of OO "patterns" (visitor, factory, etc.), that are working around the lack of this capability.
KenoFischer··on Gaining control of BMC from the host processor
It's for reasons like this that I'm very thankful that the OpenBMC effort is getting off the ground. The prior lack of documentation and understanding of what the communication channels, and how they're configured is a huge problem. Perhaps it's time to get the SPI programmer out again and see if I can't finish replacing the BMC firmware on a bunch of my boards.
KenoFischer··on Gaining control of BMC from the host processor
FWIW, in my experience there's lots of middle ground between the host being completely untrusted and the user having root on both the host and the BMC. Anywhere from "we're letting random third party collaborators share our machines" to "it's only in-house users, but they may not properly secure their machines" is a use case where regular users even with root on the host would not have root on the BMC.
KenoFischer··on High Performance Numeric Programming with Swift: Explorations and Reflections
It's a bit of a JIT party trick by emitting the target address directly rather than going through the PLT.
KenoFischer··on High Performance Numeric Programming with Swift: Explorations and Reflections
The GPU backend is (and so's the CPU backend, but that's currently too slow to get much information from). The TPU backend isn't (which is why we targeted XLA in the first place).
KenoFischer··on Train TensorFlow models faster and at lower cost on Cloud TPU Pods
Well, you lose some performance to distributed memory overhead of course, plus the TPU measure for that number is a previous generation TPU. The new ones are a decent bit faster. Plus you get four chips per "Cloud TPU". Putting all that together, TPUs are certainly a very competitive offering.
KenoFischer··on Train TensorFlow models faster and at lower cost on Cloud TPU Pods
Julia runs on these ;). Well, it will once multicore access for non-TF frameworks is public.
KenoFischer··on Ask HN: Do you use Julia outside of technical computing and ML?
I use it for systems work (compilers, debuggers, etc.), but I may not count ;).
KenoFischer··on Building a Language and Compiler for Machine Learning
We've been meaning to run this comparison, but haven't gotten around to it yet. I expect it to work and am hoping to see some performance benefits. It should be fairly straightforward to see it working, the only reason we haven't tried so far is that we only have xrt hooked up and the TF infeed ops are not open source, so the existing code doesn't just work. It should be straightforward to hook up the xla service instead, but it's a bit of additional code to write that we haven't gotten to.
KenoFischer··on Building a Language and Compiler for Machine Learning
Yes, Julia tends to be quite good at this kind of thing. Which level you want to operate at will depend on the details of the accelerator. Happy to give some pointers if you can give me a rough idea of the target architecture and what software already exists. My email is in my HN profile.
KenoFischer··on Deutsche searched in money laundering probe over Panama Papers
You're correct (inside Germany at least). db.de is the Deutsche Bahn.
KenoFischer··on Automatic Compilation of Julia Programs and Generic ML Models to Cloud TPUs
As seen in Jeff Dean's twitter feed [1] ;). But seriously, this was a lot of fun. We'll have the code up very soon - just need to clean up a few things.

[1] https://twitter.com/JeffDean/status/1054951415339192321

KenoFischer··on Compiling Julia to TPUs [pdf]
The XLA:TPU compiler is not open source, so it's anyone's guess (unless you work at Google), but my understanding is that it does NOT use LLVM under the hood.
KenoFischer··on An Introduction to GPU Programming in Julia
Sure, but that's a completely separate thing from what Julia is doing here. The whole point of the GPU backend is that you get almost the full flexibility of the Julia language at your disposal, without having to fall back to C code. As a result, not only is your code more familiar to Julia programmers, but you also get to share large amounts of code between CPU/GPU/TPU. If all you're doing is super small kernels that may not matter much, but GPUs are so powerful these days that it's perfectly sensible to offload rather large programs that you may not want to write in C anymore.
KenoFischer··on An Introduction to GPU Programming in Julia
Am I missing something? I was excited about other languages doing the same as Julia, but the first example for ClojureCUDA has

    (def kernel-source
          "extern \"C\"
             __global__ void increment (int n, float *a) {
                   int i = blockIdx.x * blockDim.x + threadIdx.x;
               if (i < n) {
                  a[i] = a[i] + 1.0f;
                }
           };")
Is there an example where the kernel is written in Clojure?
KenoFischer··on Parallel Supercomputing for Astronomy with Julia
> But will Julia be able to output the necessary instructions for the future hardware accelerators, which could be totally different architectures? I'm thinking of all the new neural networks cores, the DSP, fpgas, and heterogeneous computing from different rival vendors. It seems Julia is deeply dependant of LLVM.

Yes. Watch HN in the next week or two for an announcement that may interest you ;).

KenoFischer··on Parallel Supercomputing for Astronomy with Julia
It's not quite that easy because of a) the noise (you're fitting distributions not values) and b) the fact that multiple light sources affect a given pixel, so you have to be a bit clever. But yes, you can get parallelism by spatial partition.
← PreviousPage 10 of 17Next →