arcfide

I don't have it anymore, but I did a live coding session in ed(1) where I edited thousands of lines of XML code as a demonstration of using ed. It's a very powerful editor, almost too powerful for my "simple tools" aesthetic.

Just a few things you can do with Ed:

Structured/Outlining Refactoring Rapid code catenation and splitting

It's one of my favorite editors. I don't think I have a video of it, but prior versions of the code were optimized for ed(1) hacking, allowing me to do structured/outline editing easily with ed(1).

1. http://www.jsoftware.com/papers/fork.htm See Hook and Fork. This was the first introduction of Trains programming in APL, I think. You can see help.dyalog.com for more information of the current implementation in Dyalog.

2. I think the tree printout of trains is a built in library feature of Dyalog APL. The dfns library (dfns.dyalog.com) has a tree printing dfns that may be of use in implementing it, and I think you can read the actual source code for that in the user command for display or the like.

I may implement a version of this for my ASTs to augment the Xml and px utilities, as it is a nice format and one that might be of use for printing out the AST of the compiler. I have not had need of it yet, though, so I'll probably get to it the next time I have to do a fancy demo for which I can prepare more adequately.

Thanks for the questions!

1. I've addressed this a bit more in the previous threads on this video and the discussion that led up to the video. It's actually less write once than other stuff. However, to go with that, it's also more disposable. That means that basically, I have a low tolerance for code not continuing to make sense to me. So if I come back to a piece of code and it doesn't make sense to me, I just throw it out, because it was clearly wrong if I can't make sense of it.

I find it's quite natural to read through the code at this point. Even if I haven't thought about a particular compiler pass for a while, I just start at the beginning, and in a few seconds it's easy to see the structure and overall approach, then I go deeper and see what's actually being done. It's exceptionally easy to work with. Part of this is also because I don't have to jump all over the place to know what the code is doing. If I forgot what a helper does, I just glance a half page up and read the definition of the helper, and I know what's going on then. I don't have to keep a documentation page open to remind myself of the APIs, or the intentions or the like, because all call sites are visible, and the definition of the code is all there for me to see. A big part of this is relying on idioms rather than black box abstractions, patterns instead of information hiding.

2. Yes, I work with a number of other people who have varying levels of responsibility for the code. A good deal of them only care about the high-level concerns, and so they either use the compiler and don't read the code, or they make little tweaks here and their to suit their needs. They don't require a full understanding. A few others do work through the entire code base and we use the codebase as our single source of information when discussing design decisions and everything else. We don't have any need for supporting docs because it's too easy to work with the code to make other design docs useful when discussing architectural elements.

As for picking it up and contributing...that depends. There's a lot of background information you need to know. That's just the reality. The code itself isn't hard, but it is relying on a host of shared understanding. This includes mainly the Nanopass style of compiler design and construction, as well as a firm understanding of array programming, together with a firm understanding of the fundamental idiomatic algorithms being used in the compiler, which are the main "new thing" for people to understand after they understand how to build a compiler and how to write array programs.

Unfortunately, most schools teach neither Nanopass style compiler design, nor array programming. So, the completely uninitiated have a few things to learn. The code itself, though, is no worse than expecting someone to understand C semantics and OOP Tree oriented programming patterns in a monolithic compiler style when working with something like Clang, or tree traversal in C over C when working in a simple compiler like PCC.

3. Actually, looks can be deceiving. I'm doing GPU programming with APL, but over a domain that is considered far outside of the standard APL comfort zone. In fact, a key research contribution of this compiler is that I have developed algorithms and a holistic set of techniques for doing tree/compiler transformations on a GPU, which was previously a very open problem. This compiler/domain would have been considered one of the quintessential examples of a domain that is not well suited to APL. In fact, that's not true, it's just not suited to APL and the traditional way of thinking about this domain.

As for things like web development and the like, you can see MiServer, which is Dyalog's Open Source Web framework. There are some Dyalog User Meeting discussions about the framework, and tutorials, documentation, and the like to get you through it.

APL isn't a perfect language, but there are very few languages pushing the boundaries of human-computer interaction with language in the way that APL does, and I think this is sad.

I think in some ways we might be forced to write programs more like APL in the future, because parallelism is simply that important to the future of computing. I'm also doing research into the early stages of learning computer science, and some initial research suggests that it may in fact be better to teach students parallel programming in an APL style first, and later transition to the more traditional stuff, rather than doing it the opposite way, which so far as just convinced people that they shouldn't write parallel programs at all. In contrast, APLers prefer parallel programming and regular transform code from sequential to parallel and back again. They tend to be comfortable with both modes of thinking, which cannot be said for most other languages in common use. Agent/concurrency heavy languages might be an exception to this general rule, but they also get a bad rap.

Finally, I'd like to make a small note about bringing collaborators onboard. Many collaborators that I've had have been either inexperienced and were learning for some other reason or another, or were experienced and had one specific thing that they wanted to fix. What often happens is that the most inexperienced spend their time, as they should, learning important basics, and don't ever work on the compiler proper. I think this is the way it should be there. The more experienced people, I often find that I fix their problem that they wanted fixed in the compiler as I'm explaining the compiler to them. Because of the way the compiler is designed, I usually find myself saying, "So, to do this, you would have to make this change here [edit] and this here [edit] and ...oh wait, that's it, okay, it's done for you. Enjoy the patch!"

Good points here. I think we agree at some level here. Putting aside practical readability, we can focus on "onboarding overhead." You're also right about community.

Obviously moving from C to Java or Java to C++ is going to be easier than moving to APL. We can both agree there, and agree that part of the reason is the relative similarity between Java and C. But it's actually much worse than it appears on the surface. Let me address some of your other points before getting back to this.

Every language and code snippet is hard to read without context. Indeed, some learning theories would argue that context is such an integral part of learning that the absorption of skills and understanding is deeply contextualized and so an isomorphic task may be impossible to someone simply because the task appears in an alien context. But that shared context is HUGE and people don't realize how much they rely on it.

APL is one of the most established languages around. It was designed initially in the 1950s, and has been actively used since then. It has always struggled with the computational community, despite its success with non-computer scientists, I argue, in part, because it fundamentally challenges, and has challenged, the perspectives of the status quo. Should APL become less APL-ish simply to become more popular? J was an attempt to preserve the ideas and fundamental concepts without using the funny symbols, about which people complained, it didn't change anything. People continued to dislike it and argue against it, because they are fighting against a challenge to their conception of how things are done. This is not so much a marketing problem as much as a group think problem. You cannot provide a new way of looking at something by changing your new way to be the old way to make people more comfortable with it. That's just a new name, not a new idea.

The syntax is a fundamentally good contribution to the state of computing, it was, and it still is. Remove that approach, or the underlying principles of Iverson's Notation as a Tool of Thought, and you are no longer contributing to the positive growth of the community's ideas. It is the fundamental idea of APL that people rebel against, but they simply use the syntax as a scapegoat.

If you read "notation as a tool of thought" it should become clear that the Elixir example you are presenting, which is one of the most common suggestions for APLers, doesn't work, and there is a reason that you don't see it around.

Syntax matters. That's a fundamental assertion of APL and its community. It's an obvious conclusion from HCI research, too. It is not possible to experience the full contribution of APL by ignoring its syntax and substituting one that makes you more comfortable. I tried. As a Schemer I did this. I realized that what I was using had none of the things that I valued APL for. By trying to utilize my own "syntax" for APL, I had lost the benefits and gains that APL provides. Syntax isn't just a method of communicating the ideas of APL, it is one of the fundamental ideas of APL. It is an expression of foundational principles of approaching communication of computation ideas.

For more on this idea, especially as it relates to your concepts of "people needing readable languages", please see some papers on direct development, a distinctly APL development methodology. Here are a couple:

http://archive.vector.org.uk/art10009900 http://archive.vector.org.uk/art10500550

Particularly the case of pair programming with users, they are direct experiential evidence countering the unreadability of APL for the new user. I have also done a small exploratory study on this subject:

https://www.youtube.com/watch?v=kIItfQJEVdM

The tentative conclusion we can draw from the above is that APL is not unreadable to the uninitiated, it is unreadable to computer scientists and programmers who are attached to their programming languages.

APL has a strong history of being readable and usable by the uninitiated non programmer. It has a horrible reputation of being liked by programmers who regularly use other programming languages.

And this brings us back to the first point. Our entire computing culture, from initial education onwards teaches us that APL's very ideas are "hard to use" and hard to understand. It's not just that APL lacks a shared context between Java and C, it's that the foundational execution model used when thinking about APL programs lacks a shared context between the rest of the traditional computing society and itself. Everything you learn in school about programming tells you that you should think this way about code, and APL proceeds to completely marginalize all of the things that you are used to considering important.

It is not hard to learn or use APL. People do so regularly and without struggle. It is hard to reject a way of thinking about the world that is constantly reinforced by C-style programming models for sequential processors.

It is not hard to read, it is hard/impossible to fit into the boxes that most programmers use when evaluating things. To use a bit of hyperbole, give up everything you think you know about computing, and then APL is easy to read/learn.

It's akin to an assembly language programmer being told to program with Agda, and that Agda really will give them a new and interesting and useful way of looking at the world. But then imagine everyone else being ASM programmers, too, and schools never teaching anything about Agda, but only teaching ASM (maybe one teaches PowerPC, one X86, but otherwise they're basically the same with some slight different instructions and syntax for their assemblers). Agda uses these weird greek symbols, and there are no instructions, and no one provides a simple Agda to ASM Rosetta stone and no one explains Agda in ASM programmer concepts so that we can understand the essence of Agda. And so on and so forth.

The evidence and research suggests that APL is distinctly hard for existing programmers to understand and learn, but not for most everyone else.

Oh wait, so you're not preserving the subsequence, you just preserving the sum. So it's a standard scan. So my original translation is correct, my subsequence preserving version is doing a bit more than this by preserving the subarray as well as the sum.

Ah, if we have to preserve the sequence:

((⊃∘⍒0⌷⍉)⌷⊢)(+\,∘⍪,\)

I believe the translation of |/0(0|+)\ is ⌈/0,(0⌈+)\

I just looked at the meaning of life. That's unlikely to work with many "hosted" APLs because of the use of eval, which is a security concern in an online REPL.

However, if you remove the execute you can still get a string version:

      ⊖⍕⊃⊂|⌊-*+○⌈×÷!⌽⍉⌹~⍴⍋⍒,⍟?⍳0

42

You can paste this into the REPL at TryAPL.org.

Personally, I wouldn't want to edit my posts on a regular basis. I consider them too verbose, but I struggle to condense them.

I'm not sure what that would look like.

Define readability.

The whole point of this video presentation was to show people how to navigate the source code, and then to decide whether they still felt the design was needlessly obtuse.

APL programs benefit from efforts to simplify just as much as any other language does. However, the current version of the compiler is more readable precisely for its concision. APL is concise, but concision itself isn't everything. Notepad readability covers more than concision.

I also disagree that shrinking code leads to less readable code. Shrinking code is usually a consequence of the application of "tricks" or micro-level obfuscatory design decisions. In cases where the objective is to shrink code at a micro-level without forcing simplification of the macro-level and underlying structure of the codebase as a whole, then it tends to mess with the semantic density and uniformity of the code, leading to harder readability. However, I'm arguing against that.

Did you listen to the above presentation and watch my demonstration of the architecture of the compiler? Do you find the architecture hard to understand? Do you find the naming conventions to be arbitrary? Do you find the structure obtuse? The presentation came about specifically to address this very claim.

If you've watched the presentation and feel that the architecture of the compiler and the design of the compiler is hard to follow, and you feel that you would not be able to begin learning the details of the code, where is it hard to understand? I welcome a counter example of any other compiler on the planet that has a simpler architecture that is easier to describe.

People make the claim that APL is "hard to grok" and they think that it's hard to grok because it's concise. I don't think I've ever met someone who understand the syntax and can read APL who believes that the code is hard to grok because of concision.

The problem here is that there is a point at which you cannot reasonably shrink your code any further. For most mainstream programming languages, Notepad readability is meaningless because they couldn't make their source code notepad readable if they tried. See my other comments on the previous threads for my thoughts and definition of readability.

Readability to those who don't "read the language" is, IMO, a bad metric. Readability should be judged by the degree to which programmers working with the source can comprehend and understand the entire system as a whole, and thereby have the best confidence of making large, wide, sweeping changes. I asserts that among the trade-off between local readability of a single function and macro readability of the whole system, the trade-off should always be made towards readability of the whole system.

The goal is to be able to work with the code, make changes to the code, and continue to keep the code moving. This means that I should be minimizing technical debt and minimizing the difficulty of making changes.

Put another way, given that my entire core compiler fits on a single computer screen of two columns of large print text in two notepad files, I would assert that it is more readable than most other compilers out there. The entire macro architecture is immediately apparent, without any necessary for additional documentation or diagrams. You don't have to ask yourself or use search to talk about where a given design pattern is used, because they are all literally there on your screen at one time.

Some people would say that this is not scalable, except that I have scaled this to a non-trivial compiler, which is designed to self-host on the GPU. Most people would have said this was beyond the capabilities of most current programming techniques and languages.

I would rather be able to see the entire piece of code than have to guess at things or have a source code that may hold my hand through more of the details, but for which I cannot keep the whole thing in my head at once.

If your definition of readability is that it is accessible to the uninitiated who know neither the language nor have experience with the domain or the programming techniques involved, I agree, a more verbose style is appropriate. That's what I'm writing a dissertation for, and guides, and videos. But people who are interested in working with a piece of code require a different readability, one that conflicts with the verbosity required too explain everything to the uninitiated.

This video presentation serves as a simple introduction to the entire working mechanisms of the compiler. It is more or less complete. The only thing it lacks is detailed expositions of each compiler pass and runtime function, which have nothing to do with understanding the architecture of the compiler or reading the code. The linked video presentation should give you everything you would need to work with this code on a daily basis and everything you need to begin studying each individual compiler pass. If it does not, or if you think it is unclear in some way, I would like to improve the code, and would appreciate some specific suggestions of where it fails to be clear.

J's ideas are terrific. Dyalog is actively incorporating the ideas in J back into APL. J's choice to use ASCII characters instead of Unicode is a historically understandable design decision that I feel should no longer constrain the APL community. Otherwise, Dyalog APL and J are my two favorite APLs.

If I can't write in APL, I would probably write it in Scheme. I haven't found anything yet that wasn't quite addressable in APL, and the one that was (writing compilers) I figured out how to do, obviously.

However, if you want to create a DSL or otherwise play with notational elements in a DSL style for which macros are the "right choice" then APL is unlikely to be your friend, because I have yet to find a way to create good macro systems on top of APL. Scheme simply wins here. However, vocabulary based DSLs (combinatory DSLs or the like) are very well suited to APL.

Outside of these, I'm actively exploring the idea of theorem proving based programming, and currently things like Agda, Idris, ACL2, or Isabelle are going to be better for this right now. Not inherently, but because of tool support. I hope to change this in the future.

Remember that the FinnAPL library expects that you know your APL environment and are able to adapt the idioms to suit.

You should also consider "Notation as a Tool of Thought" by Kenneth Iverson (his Turing Award lecture) to be required reading.

The Mastering Dyalog APL book is available as a free PDF, too. Though the paper version is more enjoyable, and serves as an excellent monitor stand in need, as well.

As a Scheme programmer, one of the first things I did when learning APL was to create a new REPL/environment in Chez Scheme called Sapling that replicated APL in Scheme using Scheme implementations of the primitives.

I quickly discovered that at least for my meager mind, there was no way to combine the two as one. I could use one within the other without trouble, but I couldn't mix the two easily.

If you're interested, the Co-dfns compiler is designed to integrate with existing languages. If your language has a C FFI, then you'll be able to integrate with Co-dfns compiled code. The workflow is basically for you to produce a namespace of the functions you want to use, and then you can compile them and link against that in your Python or C or C++ code. You can then just call those functions with the appropriate arrays and things are a go. The main then you need to do is write a function to create a Co-dfns array to and from the data types you are using in your program.

Oh, another thing I thought about, the BSD people probably won't like that it's AGPL dual-licensed (commercial licenses available). Unfortunately, that's the only reasonable license for this right now. One day, if the project is successful enough I'd like to see an ISC licensed version, but that requires a good business model around the ISC license for compilers.

Actually, I'm an old school OpenBSD user who still programs with ed(1). Check misc@ out back a number of years. I've had to move away because OpenBSD isn't the greatest platform for doing high performance GPU research. :-(

The compiler is already portable to BSD. However, the underlying library I use to do GPU programming (ArrayFire) is not ported to BSD, though it Is for Linux. The Dyalog interpreter is available for AIX as well as Linux, and that's what you need to host the compiler right now. If Dyalog received the interest, it would be pretty straightforward to port Dyalog APL to BSD.

If one wanted to provide a self-hosting Co-dfns compiler, you could implement a minimal dfns interpreter and then run the Co-dfns compiler over that. To my knowledge no such interpreter exists yet. However, porting either ArrayFire or the Co-dfns runtime would go a long way to doing this, because you could then share the runtime library between the interpreter and the compiler.

So, yes, this is quite possible, and probably not even that much work. I'd welcome such a contribution. :-)

I cover how I debug my program in the video. I use two utilities, pp and px that are identity functions that print their results as side-effects. I'm happy that I'm working in a non-lazy language.

This compiler is poetically driven, but it is still a commercially funded and commercially marketed product. It's just that in the APL community, you can pursue both and have them be compatible with one another.

In particular, the desire to make the code as "disposable" as possible has lead to me using strictly combinatory/points-free style programming in the core compiler. It makes it very easy to change things because it's easy to see the whole picture and easier for me to delete any given piece of code. Many "fixes" in my code are single character APL fixes, or maybe two character APL fixes.

One reason for not being able to stick to the same core abstraction throughout your whole code is that you may have an inadequate core abstraction. Part of the design goal in the compiler, which I am trying to point out in the video, is that the design is meant to keep you in the same "context" or abstraction level throughout, so making a change will always fit the current compositional context. I do this by focusing on simplification as much as possible. I also keep the code as stupid as I can get away with.

This whole discussion got started because of the amount of change that has happened with this code base (hundreds of thousands of lines). You can read more about that in the other historical threads referenced by dang. I've rewritten the compiler at least five times.

This need to make lots of changes all the time and adapt means that I actually push more towards the points-free, not away from it. By sort of diving head first into it, I can gain the benefits, but lose the disadvantages that happen in the "middle ground" where you are jumping back and forth between points-free and not-points free.

The complexity of this compiler is about the same as what you would normally encounter if you were going to write a core compiler for Scheme (not the syntax expander). It's perfectly suitable for procedural programming languages, or any other, but obviously you'd need to tweak things. One of the general strategies in the compiler is to eliminate as much irregularity as early as you can get away with. A lot of it depends on how neatly your underlying IR/ASM language that you're targeting matches the semantic needs of the language you're compiling. For example, I don't demonstrate how to do tail call optimization here, but if you are familiar with a Kent Dybvig style tail call optimization pass such as found in Nanopass compiler examples, you could adapt those ideas to this data-parallel style.

Basically, if you already know how to write a Nanopass compiler, the transition is easier. If you are used to writing monolithic compilers such as the clang front end or PCC, it will be harder, since you have to absorb the Nanopass style of thinking.

As for how usable the compiler is, that depends on what you mean. If you mean, is the compiler usable right now as a product, then it covers most of the bases for functional array programming, but if you see the LIMITATIONS file, you'll note a few things that it's missing. Ironically, things like if statements and a lack of recursion are important, but much less so to a dfns programmer than having the right primitives already available. As a dfns programmer myself, I almost never use recursion, for instance, though it's good to have it when you want it.

The next few releases of the compiler are focusing on eliminating some of these last issues of usability, but for the most part the compiler is already completely functional.

As for whether a dfns programmer could readily get working on the code base, the answer is yes and no. They would have no problem reading the code, as it's dead simple to a dfns programmer. The overall structure would also be easy to see. For them, the trick would be understanding the "big picture" of how a compiler works. Most dfns programmers are not also compiler writers. Understanding the Nanopass strategy of compilation, and then understanding why a given pass is there in the compiler, and mapping a given set of array operations (which they would easily understand) to the domain space of "compiler operations" would be the struggle. Sort of the reverse. The compiler writer would struggle to understand how compiler operations map to array operations, and the array programmer would struggle to perceive the larger meaning of the array operations in terms of traditional compiler concepts. Printing out the AST throughout the whole compiler (easy to do by editing tt) goes a long way towards aiding that intuition.

Once the core idioms used in the compiler are understood, the rest is pretty straightforward.

Among the APL language families I would recommend are Dyalog APL and J. J is great for some of the ideas it has implemented and to gain a flavor of Iverson's vision of APL as it evolved. Dyalog APL is, IMO, more practical, and we are quickly integrating as much of J into APL as is reasonable.

Personally, I am highly biased towards using the APL symbols, and not to get "sucked in" to the idea that using ASCII symbols is easier. I think the APL symbols provide important visual and pneumonic information that actually makes learning better over time than trying to use ASCII. There is built-in APL keyboard support on Linux, and Dyalog APL comes with a good IME for Windows and Mac, so there's no reason not to use the symbols anymore.

I don't know that you should avoid any of them, but I would encourage you to use dfns if you can. Using a lexically scoped syntax is going to give you a better idea of how "good modern" APL is written. A lot of APL's don't include the J innovations or they don't have structured programming features or lack lexical scope, which is a bummer.

So, overall, go Dyalog. I'm highly biased.

1. There was a lot surprising throughout this whole process. For a while I didn't think it could be done. That it could be done was surprising in itself. That using trains actually improved my life was also surprising. There are some "research class" algorithms that I have published as a result of this work, so, there are some new idioms and algorithms that came out of this work. Additionally, this whole compiler is essentially a new treatise on the construction of compilers in general, providing a set of idiomatic methods for accomplishing tasks that would have been traditionally accomplished with other algorithms more commonly taught (such as the way you would normally do a depth-first tree traversal vs. the use of a path matrix to compute parent-child relationships and using key to transform the tree). Some compiler passes turned out to be really surprisingly easy, such as function inlining, which is a recent addition not shown here. After being familiar with this code, it took very little time to explore, write, and implement. Certain things that are multi-part algorithms in other languages turn out to be a single, well-known APL primitive, such as Key (for tree transformation) or Tally for AST/tree "count".

2. It depends on how you write your compiler and what your core abstraction level is. The compiler design here is language agnostic, and you could implement a compiler for any dynamically typed, lexically scoped programming language using this compiler as a template. Adding a type-checker on top would not be difficult, but I haven't established what I consider to be "idiomatic" type-checking algorithms yet, though I did have a version of type inference that was okay that was only recently removed.

However, chasing concision or using micro-level tricks without understanding the macro-level issues being addressed by the micro-level tricks will probably lead to unreadable code. The main point here of the compiler design is the importance of prioritize the macro level thinking and optimizing the micro-level to support better macro-level reasoning. How you can apply this to your own compilers depends on how much you're willing to alter the core "aesthetic/HCI philosophy" of your code base.

3. It would be almost trivial to do so, and would have probably zero pedagogical value in the way you're thinking. To do so you would change the "gc" function to spit out Scheme instead of C code, which would be trivial, and it would require changing a bit of the syntax of the outputted code on the following pages. If you wanted to reimplement the runtime as a Scheme library, that could also be done. You'd lose parallelism, of course. The C code and the Scheme code would probably be equivalently clear.

The reason it wouldn't be much use, is because there would be nothing idiomatic about the Scheme code upon which you could "rest your hat." Reading APL written with S-expressions rather than APL notation is harder, not easier. It looks no more intuitive to a Scheme (I was a Scheme programmer before this [see R7RS Small]) programmer than the APL.

Keep in mind that the core compiler is literally just a huge string of function calls. It's literally nothing but function composition.

(2=+⌿0=X∘.|X)/X←⍳20

vs.

(let ([X (apl-iota 20)]) (apl-filter X (apl-equal 2 (apl-reduce 1 apl-plus (apl-equal 0 (apl-outer-product apl-modulus X X))))))

Now, if you're really familiar with Scheme, then the terms iota, filter, and reduce should suddenly pop out at you, but that's dangerous, because Scheme's implementation of these don't at all behave the way APL's does, though they are in the same general algorithmic family. So, this might help your study for about 1 hour, before you had memorized all the most common operators for which Scheme has a similar primitive that kind of does something a little like it. However, we just about covered all of them in the above code.

Given the above code, what does it do? Was the Scheme significantly easier to understand? Now what if you had 90 such lines? The benefit of such translation starts to die out about after the first line of APL code that you read.

And actually, at some level, it is. If ASM is the right level of abstraction for your core problem, using ASM to solve the problem is probably infinitely more simple than using Scala to implement ASM semantics.

But saying that you can easily edit ASM in Notepad is not what I am saying above. If you write your Scala code correctly, then with some Scala code, you can easily write it with Notepad as well.

However, as you continue to add complexity, ASM begins to be the wrong core abstraction, and begins to have usability issues that make it very difficult to edit in Notepad. How would you easily deal with refactoring in Notepad with ASM? Changing a single line Scala function declaration is probably a lot easier in Notepad than changing 10 - 20 lines of ASM enforcing a given calling convention with various elements. Thus, many people who write ASM make heavy use of macros to improve the usability of the syntax.

However, it is fundamentally easier to edit a single line of ASM than a single line of most of the Scala I've seen. But the ASM line isn't doing enough to make that editing efficiency pan out.

It's not a single absolute, it's an aesthetic and a design trade-off happening at multiple levels that has to balance a number of factors that are not easy to measure. However, the simplicity of editing is a secondary metric that can assist in this, the same that line count or token count can help us to understand the "complexity" of a code base.

To take a more specific example, as the distance between call sight and definition sight of a given name increases, it becomes increasingly difficult to use Notepad to make edits surrounding those two related points in the code, because Notepad is a poor tool for traveling far distances through code. Thus, notepad and other editors like it will generally encourage you to avoid putting your call sites and your definition sites so far apart from one another in the code base that you cannot get from one to the other easily.

In my case, the distance between call site and definition site in my core compiler is usually less than 10 lines, and more normally just a single line. There are a few that would be one click/page away.

So again, I'm not saying that you can determine code complexity solely from Notepad ease of editing, but that Notepad has a simplifying effect on the code base, and is an useful metric for evaluating complexity.

And as for your specific example, if you take a programming written in Scala, and a program written in ASM, that do the equivalent things, I think most people would find the Scala program easier to edit with Notepad than the ASM program, until the point where the ASM program is very small, at which point the ASM program would become the favorite, most likely.

In 2017 I can mostly only recommend Dyalog APL. Keep in mind that I am highly biased because my compiler is written in Dyalog APL, and they sell my compiler commercially. However, IMO< Dyalog APL is the place to start with APL in 2017.

They now have Mac/Linux versions that include the "RIDE" (remote IDE) which allows you to work with the code similarly to the IDE/Interface you see me using in this video. I don't use any of the IDE features, such as the code editor, debugger, explorer, or the like, but for a new user it will be quite useful, and it will allow you to "point and click" your way through, including the entering of the APL symbols.

Dyalog APL for Linux also has a slick, but old school console IDE, which includes a built-in full-screen text editor, debugger, and the like, which is built around console interactions. If you're a hard core CLI guy, then this may appeal to you.

There is an excellent book called "Masterying Dyalog APL" which covers all the major big features, built-in frameworks, and the like, for working with Dyalog APL. Dyalog comes with built-in interfaces for things like R, .NET/Mono, ODBC/Relational databases, sockets and networking, cryptography, &c. It also ships with things like vecdb, a sophisticated web framework, and a GUI framework (windows only).

Additionally APLers should all have a copy of the FinnAPL Idiom Library at the ready. It serves as the "core library" for a lot of common tasks which takes the place of "example app" and "library calls" in many cases. Additionally, Dyalog has a large suite of dfns to use that they document at dfns.dyalog.com. Dyalog also comes with document processing and graphing libraries that can be used interactively or programmatically to produce all sorts of documents and graphs.

My general recommendation would be to stick to using dfns as much as you can, programming without trying to use external libraries. It's a common "mistake" that people learning APL try to "find library function to do X" when it's often a "half a line" APL program that would be considered "idiomatic" in APL and thus not something APLers would have a "library function" for. It's better to master the core language, because you'll find your need for libraries diminishes quite a bit once you become comfortable with the language. Not that there aren't libraries, but if you really need a library function, you'll know it, and then you'll know if you can find it.

The best example apps for writing dfns would be the dfns library by Dyalog at dfns.dyalog.com which comes shipped as a dfns workspace for use by Dyalog APL programmers. These are well commented, and cover a vast array (snicker) of topics.

Function trains are one of the neatest features of the Dyalog APL language, and represent an extension of traditional APL 2 into the "Rationalized APL" space from which comes the likes of J. The example is the above video. The 90-line compiler component that I discuss in the above video is written almost exclusively as a function train. I believe I dedicate some time to talking about function trains in the above video.

You cannot do these things in Scheme, as they are fundamentally a syntax that is incompatible with S-expressions. You could do a form of points-free programming in Scheme, but the nature of S-expressions means that the structure of the point-free code is explicit, and thus, if you wanted function-trains, you would have to implement a macro to construct them for you. That could be done, but at that point you would be well on your way to implementing APL in Scheme, rather than using Function-trains in Scheme.

Function trains also don't work well for functions whose arity exceed 2.

dfns is a syntax invented by John Scholes for implementing the APL notation using a lexically scoped syntax for function definition and control flow. They replace the more traditional flat namespace programming that is dynamically scoped, or the OOP programming model, both of which are still supported and widely used in Dyalog APL. However, to a Scheme programmer, dfns is a God-send, because they closely mirror the core constructs and concepts of a Scheme programmer, namely: lambda, recursion, and if statements/exception guarding.

When I don't use function trains in my compiler discussed and demonstrated in the linked presentation, I'm using the more explicit dfns syntax.

Going further with this idea, at the beginning of this video I point out an important "metric" that I use to evaluate simplicity of the code. This is the ease with which I can manipulate the code base using "ascetical" tools. Namely, your operating system's equivalent of Notepad. In the case of Unix, it would be Ed (not Vi, but Ed), and on Mac it would be TextEdit. In this presentation I use Notepad.

We can define "easiness to grasp" along one dimension by asking "how much tool support is required to work with the code?" Code that requires, for example, a Visual Studio IDE to manipulate and work with effectively could be argued to be more inherently complex or less easy to grasp, than a code base that can be worked with easily using only Vi (the original Unix version, say, or Vim, if you want to get fancy).

A large part of this code (over time) was written, debugged, and programmed using Ed, although some of that code has since been deleted and replaced with newer versions. You can ask yourself, in your "easier to grasp" syntax, would you feel comfortable restricting yourself to using Notepad to edit the code? What about Ed?

I think most people would agree that they may go insane if they had to use Notepad to edit their code or work with it as the exclusive means of interacting with the source code. They rely on other tools besides just the syntax to understand the code and work with it.

Because one can easily manipulate this code without strain using a simple editor like Notepad, I would argue that in at least this one dimension, it is easier to grasp. Change to some of the other commonly preferred syntaxes, such as S-expressions, M-expressions, or long names for the primitives, or non-points-free style, or the like and I think you'd be wanting a more sophisticated editor.

This question has been presented to me on this research, and this presentation I am giving in the video above is actually a demonstration implicitly addressing just such an issue. If you watch the video you'll see some examples of how the "one lining" features of the language assist me in exploring and discovery. I also go over how I leverage the use of the concise nature of the notation to ensure that my entire compiler is visible on the whole screen at once.

You can ask yourself, after seeing how I put this compiler together and how it is architected, whether the features that I consider important here and how they help me work with the code, could be achieved using a different syntax. In fact, the code for the core compiler is only about 90 lines long, so you could just take that code and invent any syntax you want and then try it out and see for yourself whether you can improve the syntax in some practical way that preserves the important elements of the design and makes it easier for people to read/learn/absorb.

As a simple first step, take the code that I demo (the 90 line compiler) in the video, and replace each symbol with its name and spaces on the left and right of the name. Then look at the code again and ask yourself if it's any easier to read. My experience is that it will be fundamentally harder to read. Not just harder, but that it will get progressively harder to read the more familiar you are with the language, relative to the other notation. It's not just harder at a constant, but that the more you work with the language, the more difficult it becomes to work with these "less obtuse" syntaxes in these situations.

My take is that, yes, people don't like APL, and are biased against it. It's a case of not understanding/knowing what they don't know. I was in the same boat, which is why my discovery of APL was so interesting.

APL on the GPU isn't really a "big deal" to most people either, because they think they could do the same thing with a library. I'm taking it a step further than that.

The biggest reason, I think, that people are slightly (but not totally) more receptive to the work I'm doing in my own research field is because I'm almost literally doing the impossible. And that's sort of the barrier to entry. Until you demonstrate that you can do the impossible, they don't take you seriously. In my case, it's producing a compiler that people have said was practically impossible to do, and doing it with APL, in such a way that it becomes evident that APL is a key contributing factor to my ability to solve the problem.

Basically, what I've done is create a compiler that is natively expressed in a format/style that is as compatible with the GPU as C or C++ is with the CPU. There is no one else who has done this, or even, to my knowledge, made the attempt. And unfortunately, it's that madness of the attempt that makes the whole thing interesting.

It's easier for me to convince people (on some level), because if you ask them to imagine how they would do this with their own language, they can't. With almost every other domain, people can convince themselves that they could do the same thing more productively in their own language, even if evidence of people trying and failing suggests otherwise.

Another dimension to this problem is that APL is usually presented over domains which seem "obviously suited" to APL, and that makes people think of it as a niche domain specific language. Working in the field of compilers is an obvious not-APL-friendly domain, and yet the results are just as compelling, and thus, more interesting to people.

Unfortunately, no. There's a lot of code demonstration, some REPL work, and interacting with the source, so I'm not sure that the transcript would communicate things adequately without significant editing and adaptation. I would like to put this into some sort of guided documentation, however, at some point in the future.

This is an AMA for the linked compiler presentation by request. It is not live video, but I will be answering your questions directly here on Hacker News. This was requested due to some logistical issues and so forth during the live presentation. The content is currently being edited to start at the appropriate place, but at the moment you may have to skip ahead 17 minutes to catch the start of the presentation.

The official live stream is up now here:

https://news.ycombinator.com/item?id=13638086

Here are some things that I'll try to remember to get to:

1. Disposable Code 2. History of the Design of the compiler, including all the other stuff I've tried. 3. My own programming background. 4. Comments in the code 5. Smaller code vs. Readable Code and Quality vs. Size 6. Abstractions and their proper use 7. The aesthetics of programming

The live session is up and running now. You can find more information about the stream and ask your questions at the following post:

https://news.ycombinator.com/item?id=13638086

This is the live stream that was talked about here:

https://news.ycombinator.com/item?id=13565743

I'll be doing this in a sort of AMA style. Please ask your technical questions here. Please use the YouTube Live chat to discuss video, stream quality, or audio questions or comments.