arcfide

This might be true for standard consumer applications that can fit within the web paradigm. However, you then fail to be able to leverage vendor specific interaction paradigms, which is something I want to be able to do, such as the Mac touch strip, the Windows Surface Dial and Pen, or the Gnome title bar/notifications/messaging interface. I like to interact with voice services, so you need something for this as well, or handling the various HiDPI features of each system.

I also don't know if I would consider C/C++ or Python or the others success stories. They all have an ecosystem around them that allows for developers to be funded doing work on these projects, but only because of other products that are out there. Other products produce money, and are making enough to allow some companies and individuals to fund core technology improvements. That's different and a harder model to work with, because it requires massive scale to be effective.

If I were to expand that idea a bit more and remark on the biggest class of bug that causes segfaults for me in ArrayFire, it would be the requirement for explicit data type casting using as() to avoid problems. I wouldn't want this to go away at the cost of performance, but....

ArrayFire was designed primarily as an user library for scientists, it seems, and so there is a clear bias there. My coopting it for use as an IR language for a compiler front-end is perhaps outside of the original scope of ArrayFire. The good thing is that ArrayFire manages to capture a few key ideas that are often absent, but critical to APL, so that was good.

On the other hand, it is non-trivial to work with more advanced concepts and workflows/algorithms in a way that allows me enough control over my data to get the performance I want when dealing with nesting or other nuanced execution patterns. As a trivial example, it would be not having more control over how Scan works. ArrayFire is unfortunately not unique in the very limited amount of support for scan operations that will be executed in parallel. I have worked around this, but it's not ideal.

Additionally, the rank limitation of 4 in ArrayFire is infuriating. Given that we're working with C++, it seems like there should be no reason that I should be able to have arbitrarily many dimensions.

The design of arrayfire also makes certain code compositions difficult (this goes back to the issue with nested execution) meaning that I'm spending a lot of effort on working around this limitation in the compiler, having to add new compiler passes and optimizations to massage the output code in such a way as to avoid the syntactic and semantic limitations of the arrayfire primitives. I'm not sure ArrayFire can do better in this regard, and this is the sort of optimization that a compiler front-end could make plenty of use of, but it would have been a nice touch to have some built-in support for mixed CPU/GPU statement overlap to avoid me needing to separate out certain codes and possibly introduce various stages to make sure that all the computation can be properly fused and executed. This limitation is especially felt with GFOR.

So, if I had my way, ArrayFire would be less user friendly to the scientists (that's why Co-dfns is here! ;-) ) and more friendly as an IR for implementing APL runtimes. :-)

We're working on it! We're getting close already. If you have a naively parallel algorithm that is good for doing CSP style decompositions on, then you can compile your individual unit programs using Co-dfns and then using Dyalog APL Isolates to distribute the bulk computation across the distributed computing cloud. It's more work than it should be right now, but it does work for the right types of problems.

I'm actually designing one better. :-) Probably a few years out from true production ready (maybe more, who am I kidding, probably more), but definition something cool.

If that works for them, more power to them! They'll still likely need a commercial Co-dfns license if they are doing that, not because of the produced code, but because of the programming environment in which they do the development, unless they want to slice Co-dfns out of its "host" environment.

You are right, however, that you can use the free, non-commercial Dyalog to run the AGPL Co-dfns. Co-dfns is both a commercial and a research enterprise, and so it is important to me that the ideas in the compiler are manifest openly to the public, hence the open-source nature of the project. However, it also needs to fund that research somehow (have you ever tried to get a grant to write APL? Hahahaha!), and so commercial uses need to be managed in a reasonable way, namely, something mutually equitable.

Some interested parties are working to use Co-dfns in public facing services, and not just private data sets. One group is very slow moving, but we're exploring the possibility of a rewrite of the cryptographic stack.

:-) Thanks, but see my talk above. I'm slowly attempting to build just such controlled experiments. My first exploratory study on such is discussed in the above talk.

😸

A quick search reveals at least 12 ideographs for cat.

Nobody wants to write proofs as English prose like they did when first proving basic algebraic properties. Yet those proofs are much easier to guess at the meaning of, even if you don't fully get it, than using our standard mathematical notation. Of course, it just so happens that our mathematical notation allows middle schoolers and high schoolers to think of these proofs as obvious and trivial, whereas they would have no idea what to make of the English proofs that are more "natural" to the English speaker. Of course, that notation that they use with such facility (even if they aren't particularly competent with it and just learned it last week) has no relation and no intuition at all to the English language, and makes no attempt to do so.

Yet, I challenge you to back up your statement about character counting by showing me a Computer Scientist and a Software Engineer who no longer writes or uses traditional mathematical notation for anything, but has systematically moved over to the more intuitive and natural programming language of their choice, such as Scheme, C, or Haskell.

APL is a suitable replacement for Mathematical notation and was designed as such. Other programming languages are not. That's a very important element.

It's not about character counting. See Iverson's Notation as a Tool of Thought Turing Award lecture.

You might have a stronger case if, since the advent of computing programming languages, which are demonstrably capable of representing a whole host of mathematical ideas and concepts directly, people have begun flocking to programming languages to replace their mathematical notation in the communication of ideas in non-computer science contexts.

However, people rarely consider a programming language equivalent to be better/superior to traditional mathematical notation when they are talking amongst themselves in front of a whiteboard.

I'm not convinced of this. See my talk:

https://www.youtube.com/watch?v=kIItfQJEVdM

One problem with cross-platform graphical environments, IMO, are that they fail the integration standards that I've come to expect from GUI apps. I want Mac apps to feel like Mac apps, Linux apps like Linux apps, and Windows apps like Windows apps. This means making heavy use of specific interaction paradigms for each platform to achieve the best level of integration. This isn't possible with shared graphical environments.

Co-dfns is Open Source, and funded/licensed by Dyalog. They also produce MiServer and a host of tools Open Source as well. The problem is that they have to fund that work somehow, and I've yet to see a good business model on interpreters and compilers for Open Source that works well enough for Dyalog to take the risk. If I can successfully market and fund Co-dfns through an Open community funding model, I think it would go a long way towards encouraging Dyalog in openness (they are already quite open to it).

Dyalog does have a Windows windowing environment, but also their web framework, for designing interfaces. They are actively working to improve their cross-platform situation, with things like RIDE, MiServer, and the like.

As long as you can live without the GUI elements of plotting, you can use the shipped Plotting and Documentation tools (SharpPlot?) that gives you graphing and plotting tools, and produces images that you can open in a viewer. This is what I use when generating graphs and plots on Linux. SharpPlot or the Plot Wizard is very nice on the Windows IDE, but the purely command line version works great on Linux and Mac.

Cross platform GUI applications for Dyalog are best done either with individually coded backends or making it a hosted web application through the MiServer Web Framework, which has some support for doing this.

Acquire the APL, and then be the hero in your own code every day.

Ah, there it is, thanks!

Thanks!

You'll need to get at least a free copy of Dyalog to use the compiler. They have a number of different free options. If you intend to use the compiler, it is available for use by all under an AGPL license. However, the generated code also uses Co-dfns code and is likewise under an AGPL license. If that license is unsuitable for you, then you'll need to arrange for an alternative license through Dyalog. Normally this will only be required for people interested in using Co-dfns commercially in their closed source projects.

The Co-dfns work is run like an Open Company however. You are welcome to submit contributions (see the README for more information) as pull requests and can onboard yourself at your leisure. I'm working to encourage more people to support the funding of Open Work on Co-dfns, but that's a long, slow process.

So, yes, generally speaking the compiler is designed to be used in conjunction with the host of additional tool support you get from Dyalog APL (the interpreter). However, Dyalog makes it as easy as possible to get a copy of the interpreter.

I try to avoid going bare metal as much as I can because of the difficulty involved with working with GPUs at that layer. I try to focus on higher level concerns, and use the excellent low level work of people such as the ArrayFire developers to handle the low level stuff. I still have to understand it and work with it at some level sometimes, but for the most part I stick to my core abstractions.

Currently Dyalog APL provides green-threaded concurrent programming using a traditional "fork" model as well as a CSP-style distributed computing model called "Isolates." In addition, Dyalog will utilize vectorized operations on the CPU for a number of primitives and code patterns. Co-dfns provides for additional functionality by providing vector level parallelism on the GPU/CPU and a true threading "fork" (this feature is only part of the language extension, and has not yet been implemented). So, if you write your code in a Co-dfns compatible manner (closed single Namespace of dfns, and some limitations in the current version which are being lifted rapidly), then you will be able to execute it on the GPU, CPU, and SMP machines.

As a trivial example, say you wanted to compute the quadratic prime numbers algorithm in parallel:

    Primes←{(2=+⌿0=X∘.|X)/X←⍳⍵}

You would simply compile this function inside of a namespace using Co-dfns and you could then run this function on the GPU to improve performance. The compiled code would run on either the CPU, GPU, or SMP machines.

When I spoke about benefits on the GPU, I was speaking of the benefits of executing the compiler passes on the GPU, because with a Nanopass architecture, or really, the Micropass architecture I'm using here, each pass is simpler than the equivalent set of passes that do the same thing in a traditional compiler. This means that the passes are generally each simpler, and thus, easier to parallelize.

In compilers the N value is commonly considered to be the number of nodes in an AST. Compiler performance is generally evaluated based on its performance over the code, which is generally bound by the number of nodes in the AST.

The identity pass is linear in the size of the AST in almost all functional compiler designs. Optimizations would then make this constant time for most functional languages with adequate optimizations. The identity pass in Co-dfns would be the single character ⊢. The Co-dfns compiler would produce code for this pass which is constant time.

Likewise, with a bit of work, this would also be true for a Nanopass compiler function, unless you insisted on traversing the AST and rebuilding it yourself, in which case the framework could not optimize this away.

The majority of compilers use algorithms that are quadratic in the size of the AST (number of nodes) for a number of passes. Many compilers use a variety of graph coloring register allocation, which is NP-complete. However, better register allocation algorithms exist that are quadratic or quadratic log in the number of nodes in the AST. The Nanopass research has demonstrated that for real life compilers, the cost of certain compiler passes generally overwhelms most of the cost of the rest of the compiler, and so adding another linear factor by splitting a pass into multiple passes, generally does not affect the big-O complexity of the entire compiler, and thus, the overall performance at scale, though it can affect the constant factors by a small amount, depending on how good your allocation engine is for your host language.

This Nanopass research has only been done on compilers implemented on the CPU. My research is exploring whole compilers on the GPU, which is an area that shows promise, since even memory bound computations such as most compiler core compiler transformations, can see significant performance speedups, even if they move from a linear to log linear complexity.

The passes are not run serially. Each pass is data parallel, and while there is a serial dependency in that each pass is dependent on the results of the previous pass, nothing in the way the compiler is written forces a hard barrier (such as a join in fork-join programming) at the compiler pass boundaries. It is conceivable that optimizations techniques for stencils such as Trapezoidal decomposition could be applied to compilers written in this fashion to allow intra-pass optimization to occur, but this is future work.

The passes themselves execute in parallel on the GPU, or at least, that is their design. They are expressed as natively data parallel, or vectorized, algorithms. If run on the CPU through the interpreter, then they will execute using the Intel Vectorized code units using the optimized primitives in the Dyalog APL interpreter.

Here's an English reading of the above code:

Filter the numbers from [2,R), removing any number that is a member of the set of products of pairs of elements in [2,R).

See Andy Keep's dissertation on using Nanopass to reimplement the Chez Scheme compiler. The short answer is, not really. Nanopass can add some constant factors, but is essentially a linear/constant addition, whereas the majority of compilers are quadratic or greater in complexity. Chez Scheme was/is N log N in most cases, and even then Nanopass did not significantly harm performance.

It actually has benefits on the GPU beyond just the CPU, because the passes tend to be smaller, and thus more readily executed efficiently on the GPU, meaning that you would likely see a large net gain in performance by using Nanopass. Since I believe this might be the only compiler of its kind, it's hard to say for sure on that mark, though.

Thanks for giving me a chance to chat a bit more!

Thanks everyone for the great discussions and interest. I'm going to get back to this thread later, but won't be actively commenting for the rest of the day. Please feel free to continue discussion and continue asking questions and I'll get to them when/if I can.

Also, feel free to contact me directly to discuss anything at a higher bandwidth level. And as always, see the GitHub repository for more information on the project and ways that you can contribute (help out, fund the project, contribute code, bug reports, or patches).

https://github.com/arcfide/Co-dfns

If I can "shill" a little bit, you can fund the Open Source/Open Work aspects of this project through micro-funding at Gratipay, which helps to support and keep the Open Source and community aspects of the project going. Dyalog is a great company, but ultimately, as some have mentioned, it's the community that helps to keep these ideas moving forward and improving.

That's an interesting idea, and I can see there being some overlap, but the Key operator is actually not my invention, but is a member of a class of operators sometimes called classifiers. You can think about the GROUP BY operator in SQL. Or Henglein's work on discriminators. Key is an operator that is relatively new in Dyalog that came from J and some other APLs, I think. I have to go back to my paper to remember.

http://dl.acm.org/citation.cfm?id=2935331

Good luck, and please feel free to provide feedback. I'm always looking for ways of improving the situation or approaching it from a different angle.

You may also want to see "Game of Life in APL" on YouTube.

https://www.youtube.com/watch?v=a9xAKttWgP4

Try this one:

(2=+⌿0=X∘.|X)/X←⍳20

:-) They're equivalent, but the second should give you a bit more intuition based on your understanding of the definition of prime numbers.

Use TryAPL.org to explore the code. Evaluation is right to left with parentheses being their standard meaning. Try seeing what the left and the right sides of the / expression give you, and then try removing pieces from the left of the left side expression of / incrementally to see how its built up.

If you need help seeing what the functions do, you can see help.dyalog.com "Language Reference" That will show you what the symbols do.

I would consider ArrayFire a "hard abstraction" that is basically the bottoming out of the system. Previous to this I was using OpenACC. See my talk at the Dyalog User Meeting Last year to see some of the struggles there with Scan.

In short, ArrayFire is handling a lot of the core GPU algorithms for me, and that's its main contribution. This prevents me from needing to implement these myself, because this is itself a full time job. Much better to put a hard cutoff point and work on the high-level questions, rather than spending time trying to support three different architectures by myself. There is basically no good way of working at that level that isn't ugly and very very painful. I am glad to have someone else do that for me.

The benefit is that I can take better advantage of their platform independent and performance tweaks for individual GPUs (which is still a lot of work with the state of GPU programming today). This leaves me to focus on higher level concerns that are more relevant to the APL programmer.