Fear and Loathing in APL (2015) | Hacker News Reader

Fear and Loathing in APL (2015)

theburningmonk.com

90 pointsjxub25 comments

I got 500 error. Here is cache: http://webcache.googleusercontent.com/search?q=cache:9IikFjT...

I'm seeing a lot more APL-related submissions in the past few months. What's going on?

There is always a place for APL on Hacker News! But the last couple days have seen a number of follow-up posts on the front page.

One APL-related submission reminds people on HN of the existence of the language. Others begin reading about it, some even try it, and a few of them post subsequent links on HN. Repeat.

People got enough of AbstractSingletonProxyFactoryBeans? I am picking on Java because I work with it the most. Anyways, I guess simplicity, terseness and conciseness are getting more popular.

I was initially fascinated by the Dyalog YouTube video where the guy codes Conway's Life in APL.[1] It have certainly never witnessed such programming before.

[1] https://www.youtube.com/watch?v=a9xAKttWgP4

I was much confused by this example:

> AND and OR semantics are represented by ∧ and ∨ respectively.

> (1 3 5 > 6 2 4) ∧ (1 3 5 = 6 3 1)

> => 1 0 1

This is an error in the article. If you type the expression in TryAPL.org, it produces 0 1 0 - as logically expected.

While it's interesting (pretty funny) that this language exists, it looks downright awful to program in. Has anyone ever programmed anything sizable in it?

Apparently yes: https://news.ycombinator.com/submitted?id=arcfide And his website: https://news.ycombinator.com/from?site=sacrideo.us I'm sure there are others, but this I happened to know from memory.

There are other languages that operate within the same paradigm, such as J, which do not make use of the specialized symbols and so are somewhat more accessible. /u/Godspiral is a Reddit user who moderates the /r/DailyProgrammer sub, and he occasionally posts solutions to the challenges in J [0]. It's still plenty difficult to read, as you'd expect haha.

[0] https://www.reddit.com/r/dailyprogrammer/comments/7t6fnc/201... (This is just the most recent non-trivial example I could find of his.)

> Has anyone ever programmed anything sizable in it?

A significant fraction of wall street operates on kdb+, which is a commercial programming environment in Q (a dialect of K), which is APL derived.

APL derived languages operate with a different idea of what makes a programming language good and there was an APL patterns presentation [0] posted yesterday. In brief, the goal is to be able to see and manipulate the entire solution to a problem at once instead of isolating subproblems into layers and abstractions.

[0] https://news.ycombinator.com/item?id=16513393

I wouldn't write a web browser in Q but I've done enough daily programming problems in it that I can see how it'd be useful for writing programs in what would be the 3-5k LoC of Python range.

Kdb+ [0], a very high-performance and very expensive proprietary time-series database, was written in a dialect of APL.

I've seen demos of Kdb+ and it is very impressive. It is the top ranking non-GPU database in this benchmark [1].

It is however not the easiest of databases to use -- it uses Q (another APL dialect) as a query language. There was an HN discussion a while back about its maintainability and long-term prospects. [2]

[0] https://en.wikipedia.org/wiki/Kdb%2B

[1] http://tech.marksblogg.com/benchmarks.html

[2] https://news.ycombinator.com/item?id=11561573

I wouldn't trust Marksblogg's tests. They focus on single-table "NYC Taxi" rides which don't represent most database use-cases.

He also claims he takes the fastest runtime of the queries over several runs, which is not how real tests are performed.

>I wouldn't trust Marksblogg's tests. They focus on single-table "NYC Taxi" rides which don't represent most database use-cases.

No, but they represent a hell of a lot of big data use cases.

>He also claims he takes the fastest runtime of the queries over several runs, which is not how real tests are performed.

Assuming he already checks for data not being on fs cache, what else would you take? You don't want to measure other load going on in your system and impacting your queries.

It is common practice to take the fastest runtime. This will get you as close to the actual runtime of your code as possible, assuming it's deterministic.

"He also claims he takes the fastest runtime of the queries over several runs, which is not how real tests are performed."

It isn't, but it should be. For any non-randomized run of the exact same work [1], by definition of non-randomized, any noise that appears in the results of multiple test runs can only be due to factors unrelated to the test being run, such as the OS decisions and such. Taking the minimum is the most honest way to factor those out. A possible exception for the first run, before CPU caches are warmed up.

[1]: Ponder that closely; a lot of the knee-jerk reactions to what I just said is handled by that; e.g., if the second time something is cached by the algorithm itself that was not cached the first time around, that's not the same work. Conceptually you can imagine that I'm shooting for the exact same sequence of CPU instructions being run on the exact same data values, though in practice that can be a tall bar to leap even before considering concurrency; as with almost everything in life if you look closely enough the boundaries get fuzzy.

You can use SQL92 as a query language (this is how people usually use ODBC with kdb), simply write "s)" in front:

    s)SELECT a, b, SUM(c), SUM(d), AVG(e) FROM table WHERE f > 69 GROUP BY a, b

but q is much easier:

    select sum c, sum d, avg e by a, b from table where f > 69

Having used both q/kdb and other databases, I prefer q/kdb.

Things I often hate about writing in SQL:

- Those unnecessary GROUP BY statements

- Too much set theory: ORDER BY

- Not enough join operators. asof queries are very difficult to write.

- Difficult to be performant

Things I generally hate about SQL servers:

- Very difficult to use cores strategically (in q I just run taskset+q on e.g. a cron job; in Pg setup is much harder requiring multiple data directories)

- Very difficult to keep things in ram (tablespace + unlogged)

- Very difficult to implement incremental async builds (triggers + replication is the best I've figured out in Pg; in q I just use IPC and let the kernel queue)

- Switching to algorithms to make a new function is ugly and almost a different language (PL/PSQL in the best cases is a horrible language, and C is not so bad but sometimes requires root).

I had the pleasure of retiring a large APL application a few years ago. Although I hadn’t spent time developing in it, I did spend a lot of time with the devs who had. I also had to understand it well enough to replace it.

I’ve worked on a lot of systems and a fair few languages in my time. I’d rank it as the 3rd worst application I’ve ever dealt with.

It’s also one of the few where I didn’t end up with a grudging admiration for the original developers. And I’ve replaced a “real time” event based system written in Sybase stored procs so I have a really low threshold.

What were the two other worse applications?

The worst, as a piece of code, was a Byzantine C++ application. It had been written with abstractions and indirections to cater for every conceivable eventuality. And it seemed to incubate Heisenbugs.

Usually, with a C++ app you can fix it incrementally but we tried and it was just impossible. Far too intertwined. In the end I wrote a replacement from scratch.

The other one was a Smalltalk system. Actually, pretty well written and documented. But once Smalltalk gets above a certain size it can become impossible to reason about. And it’s very hard to replace parts. Sadly, this one was a failure. It never got fixed or replaced. Just went out of business.

I would love more particulars about this.

Yes, someone wrote an APL-to-ArrayFire compiler in APL.

https://news.ycombinator.com/item?id=13797797

It's really pleasant to program in APL once you get used to it. I would put it on the same level of ease and pure fun as SNOBOL and Haskell, which are both really fun and productive to work with for me.

And someone else wrote an APL interpreter in Lua:

https://github.com/dlaurie/lua-apl

Upvoted to compensate down-votes. While this is an opinion that tends to be repeated in APL threads and can seem ignorant. It it also the initial reaction many of us have had.

Presenting it with a reasonable follow up question like this gives a perfect opportunity to provide examples of it being used and the reasoning behind why it might not be as terrible as it might seem at first.

Down-voting such comments only discourages conversation IMO, what about giving a good answer instead. (or in this case, just link to arcfide and his interesting explanations about how he uses Dyalog to be productive.)

I guess you won't know until you tried. I got to know APL when got introduced Q and KDB+ by accident and I could not believe that you can be so terse with programming while having an unmatched performance. There are entire industries relying on Q (which is heavily influenced by APL).

Reply on news.ycombinator.com