Space Invaders in Perl 6
ttjjss.wordpress.com
ttjjss.wordpress.com
> Parrot can run this, but awfully slow and JVM has a bug
> that makes it impossible to run the game.
Perl 6 isn't finished, but it's much further ahead than one might think ... http://rakudo.org/
Top article there right now is "A useful, usable, “early adopter” distribution of Perl 6". Sounds lovely, doesn't it?
The problem is that these "early adopter" distributions of Perl 6 have been released regularly since mid-2010. The first "early adopter" Perl 6 book was released in 2003.
Generations of Perl 6 hackers have joined the project, re-written a key piece of the infrastructure, broken a bunch of APIs, and left again. If it hasn't happened already, we will soon see the CHILDREN of Perl 6 hackers born during Perl 6 development start doing half-baked infrastructure rewrites.
I don't expect the project to ever converge on a stable release. Early adopt it all you want, but don't claim it's "further ahead than one might think".
Perl 6 is nearly 15 years old now. There have been many attempts to implement it during this time, and none of them have really been usable like Perl 5's implementation is.
I can recall hearing about Rakudo Star, Pugs, Niecza, v6.pm, Yapsi, and Sprixel. There are probably others that I've forgotten about. And that's ignoring NPQ, Perlesque, Parrot and all of the other intermediate implementations and runtimes that have been dabbled with.
There has been a lot of wheel-spinning when it comes to getting Perl 6 implemented. It's like they've tried everything except the proven approach of just writing a plain interpreter using C, like we've seen work well for earlier versions of Perl, Ruby, Python, and so many other languages.
I agree. It's taken 4 years to get to this latest Star, on top of P6 being 10 years old when the first Star came out. It's all way slower than most (all?) folk expected.
This has impacted early adopters, perception/marketing of P6, perception/marketing of P5, and attracting contribution to P6.
> Generations of Perl 6 hackers have joined the project, re-written a key piece of the infrastructure, broken a bunch of APIs, and left again.
There's definitely been a lot of churn.
However, if you focus on the P6 projects that still matter today (eg Rakudo, NQP, the test infrastructure, MoarVM, p6doc, the modules infrastructure) then you find activity that doesn't -- and never did -- match your arrive-rewrite-break-leave pattern.
> If it hasn't happened already, we will soon see the CHILDREN of Perl 6 hackers born during Perl 6 development start doing half-baked infrastructure rewrites.
The projects that still matter today don't allow new devs to be unduly disruptive. (Getting a commit bit to the most important bit, Rakudo, requires evidence of competence followed by signing and submitting a formal application.)
> Early adopt it all you want, but don't claim it's "further ahead than one might think".
It sure still isn't ready for most use cases. But have you given a recent release (especially the latest Rakudo Star) a serious look?
---
(Edit: Removed the opening sentence "No. " before "The projects that still matter today don't allow new devs to be unduly disruptive." because you can't stop volunteers doing whatever they want.)
Correct me if I'm wrong, but isn't MoarVM less than two years old? That would still put it squarely into the rewrite-break phase of the cycle, so it's too soon to tell whether the "leave" phase will follow or not.
> Getting a commit bit to the most important bit, Rakudo, requires evidence of competence
The few Perl 6 contributors I know were highly competent. The problem was IMHO not lack of competence, but almost the opposite: A highly skilled and productive contributor would come in, implement most of their bold new vision, but run out of steam before the new component had fully stabilized.
> have you given a recent release (especially the latest Rakudo Star) a serious look?
Not really, I'm afraid. The code examples I've seen don't really look appealing, I have not seen anything that truly made me want to try the language (though I keep hearing good things about the supposedly amazing parsing support), and the lack of a plan to ever move to a product quality release is a major turnoff.
> I have not seen anything that truly made me want to try the language
Ah. That might explain what's going on here. Presumably your view of P6-as-a-project is like your view of the language -- ie you haven't explored it in detail and are instead trusting some combination of what you've heard and what seems reasonable to you.
> isn't MoarVM less than two years old?
Coding on MoarVM began just over 2 years ago.
> it's too soon to tell whether the "leave" phase will follow or not.
I was tempted to dismiss this as a "when did you stop beating your wife?" point, but let's see if we can't just drown the witch anyway. :)
First, it must be a mistake to think that project age is a way to determine whether or not this purported P6ian "leave phase" applies to a project.
Rakudo is about a decade old. 7 of the top 8 all time Rakudo contributors still participate almost daily. (I explain Patrick's absence below.) None of the other 100+ contributors left a mess (and many still periodically hack on it). So Rakudo clearly has not yet suffered from this damaging "leave phase". But, and here's the reason why I'm saying this stuff: do you therefore think Rakudo will never be subject to the "leave phase"?
Imo what matters regarding contribution patterns is more like how complete a project is, whether a given contributor leaves it in good shape so others can understand and hack on it, and whether they are likely to. The project's age is immaterial.
Second, devs will leave projects no matter what. I fully supported Audrey choosing to radically dial back her Pugs efforts when doctors told her it could kill her; Patrick, the P6 project manager, choosing to focus on his wife and then children before and after his family's loss; Larry periodically backing off to focus on defeating a serious health problem (most recently cancer). This sort of thing can happen to any project. (Probably in proportion to the length of time the project runs.)
Now to some MoarVM specifics.
MoarVM is already a radically cleaner/leaner/meaner VM for the NQP compiler toolchain than Parrot. (This was the justification for starting it.) The current MoarVM devs could all leave today and we'd be in a vastly better place than we would if they hadn't built it. Surely this (extremely unlikely event) couldn't be classified as a P6ian "leave phase" curse whether we were 2 years or 10 years in to the project?
> The few Perl 6 contributors I know were highly competent. The problem was IMHO not lack of competence, but almost the opposite: A highly skilled and productive contributor would come in, implement most of their bold new vision, but run out of steam before the new component had fully stabilized.
That doesn't generally apply to the core projects that are relevant today, the ones I highlighted.
To briefly mention MoarVM again:
jnthn, the lead MoarVM dev, has been a lead P6 hacker for just short of a decade. One could argue Nicholas Clark has been on the case even longer.
Whether a hacker leaves a mess behind is about the character of the hacker and the nature of the project's management. While I'd agree that Pugs development was arguably too anarchic, and Parrot's was arguably too bureaucratic, the projects I've highlighted are well managed and producing results upon which others can continue to build.
That said, some devs will inevitably leave. I am very disappointed that diakopter is distracted by startup fever because it means he won't have time to build the libperl based P5 interop that's a critical part of Perl's future. I guess someone else will have to do it. Such is life in the FOSS dev world.
I'd say I've kept somewhat closer tabs on the project than on the language (partially to figure out whether it was actually time to start paying attention to the latter again).
>> it's too soon to tell whether the "leave" phase will follow or not. > I was tempted to dismiss this as a "when did you stop beating your wife?" point
It's more a case of the 5 times divorcee insisting that the latest marriage is finally built to last; it's not that I doubt the good intentions…
> The current MoarVM devs could all leave today and we'd be in a vastly better place than we would if they hadn't built it. Surely this […] couldn't be classified as a P6ian "leave phase" curse whether we were 2 years or 10 years in to the project?
It would depend on what would happen afterward. The pattern so far has been "Component X is beyond repair. Let's toss it and start over". The curse is not people leaving, it's about preferring rewrites to evolution.
> Whether a hacker leaves a mess behind is about the character of the hacker and the nature of the project's management.
I'm more concerned about the latter, and more concerned about the overall management than the subprojects.
> Pugs development was arguably too anarchic, and Parrot's was arguably too bureaucratic
But what they, to me, had in common was that neither seemed to be focused on making a practical Perl 6 release happen.
Pugs: Setting aside the question of whether the choice of Haskell restricts the pool of available developers, was there ever a realistic prospect that a production level Perl 6 would ship as a tarball with an INSTALL file that said "1. Install a Haskell compiler" ?
Parrot: Deciding 12 years into a project that the underlying VM wasn't a particularly good match for the language (7 years after the original designer more or less said so: http://www.sidhe.org/~dan/blog/archives/000435.html ) strikes me as a massive lack of a coherent vision for the project.
> That said, some devs will inevitably leave. […] Such is life in the FOSS dev world.
Not just in FOSS. But the way to deal with that, IMHO, is to plan in manageable milestones. I just don't see evidence of a true release convergence mentality emerging among Perl 6 developers.
Heh. And here was me thinking Camelia's 20 year old brother was still way too embarrassed to be seen out and about with his 14 year old sister because she was so immature!
But yeah, once bitten twice shy is an understated version of many folks' quite reasonably skeptical attitude toward P6.
> The pattern so far has been "Component X is beyond repair. Let's toss it and start over". The curse is not people leaving, it's about preferring rewrites to evolution.
If a component is indeed beyond repair, it should be tossed and rewritten.
But you are suggesting that components get rewritten that are not beyond repair. I don't agree that that applies to the core P6 projects (Rakudo, NQP, MoarVM, etc.)
> Deciding 12 years into a project that the underlying VM wasn't a particularly good match for the language (7 years after the original designer more or less said so: http://www.sidhe.org/~dan/blog/archives/000435.html ) strikes me as a massive lack of a coherent vision for the project.
You're not giving credit where credit is due -- credit for doing exactly what you recommend too.
First, surely you realize that Rakudo devs read what Dan wrote? And that they directly experienced the poor fit he spoke about? Yet they didn't give up on Rakudo or Parrot. They evolved Rakudo and its fit with Parrot hoping they could avoid having to write another VM. Isn't this all exactly as you (say you) would recommend?
Second, when the Parrot project stalled for a year the Rakudo team started MoarVM. This was also a good thing, not an unnecessary rewrite.
> plan in manageable milestones.
What do you think shipping Rakudo Star with support for MoarVM at the start of May was if not a milestone?
> I just don't see evidence of a true release convergence mentality emerging among Perl 6 developers.
If you mean something akin to "let's ship a production release!" then no, that's not where things are at.
Yes it is, but from the accounts I've read, during that time, Rakudo and Parrot were always run by different teams that had a limited understanding of each others' perspective. Technically, moving the projects into separate repositories and onto different change schedules seemed to have worked out badly.
>> plan in manageable milestones. > What do you think shipping Rakudo Star with support for MoarVM at the start of May was if not a milestone?
I was thinking in terms of production release milestones. The nearly 4 years or Rakudo Star releases demonstrate, to me, that regular development releases, in themselves, do not help production release convergence all that much.
The P6 design team had been having weekly phone calls for years. Patrick (Rakudo pumpking) and Allison (Parrot architect) were both regularly on the calls. As you might expect from any kind of regular meeting, there were occasional disagreements, but the purpose of those conferences was to guide the convergent development of both projects.
I used those calls to figure out what I should work on within Parrot to help Rakudo.
Technically, moving the projects into separate repositories and onto different change schedules seemed to have worked out badly.
From the outside, that probably looks like the point at which the projects really started to diverge, but I think it was a symptom. Near the start of Parrot, a lot of people thought P6 was an ambitious project that'd never amount to anything and they wanted Parrot to succeed with or without P6. At some point, enough people within Rakudo decided that they didn't want to be tied to Parrot forever that they wanted an alternate backend.
Aiui Patrick read Larry Wall's 2001 Apocalypse #1 (http://www.perl6.org/archive/doc/design/apo/A01.html) and took it into account before he started Rakudo. It includes the following statement:
> "Multiple backends are a necessity of the world we live in today. Perl 6 must not be limited to running only on platforms that can be programmed in C. It must be able to run in other kinds of virtual machines, such as those supported by Java and C#."
That statement didn't require that any given P6 compiler be multi backend, but I would guess Patrick was thinking something like the following about P6 and Rakudo (first, my guess about his thoughts when he started it; then, about his thoughts a year or so later):
* Parrot would hopefully be a good host for a P6 compiler at the start and remain that way; OR, if not, it would evolve to become a good host (ie the convergence which you mention).
* After Rakudo on Parrot was "done", some P6 compiler, possibly Rakudo, would eventually be ported to JVM, CLR, and other existing VMs, as tuits and priorities allowed.
Later (2005?), as it became clear that Parrot was not a good fit architecturally for P6's / Rakudo's needs, he presumably responded. I'm guessing his thinking was something like this:
* Parrot and P6/Rakudo might sufficiently converge. This would avoid Rakudo having to focus, before an initial implementation was sufficiently "done", on porting. So Patrick should encourage convergence as best he could.
* Parrot and P6/Rakudo might not sufficiently converge. This would be an existential threat to Rakudo. So he should have developed a plan B. It should have been obvious to him that this would take several years and would best be avoided if at all possible, but he should have had a plan B.
Fast forward to 2009:
> "Putting Perl on JVM was always a kind of thought ... but was never really serious until summer of 2009 ..."
This is from the first minute of the video of Patrick's YAPC::NA 2013 talk "Perl on JVM". I found the whole talk interesting but the first 10 minutes are particularly instructive in relation to the things we're discussing:
https://www.youtube.com/watch?v=XgPh5Li3k4g&list=PLRuESFRW2F...
You're spreading misinformation, and you've been told this enough times that it seems deliberate.
Imo, all is fair if you predeclare ("mere armchair analysis").
> do the hard research
I've spent a colossal amount of time reading P6 and Parrot mailing lists, IRC logs, etc. I'd estimate more than a thousand hours since 2000.
> sixperl call summaries
I've looked for an online transcript or summary of sixperl calls before. I'm failing to find them with search engines now. Would you be willing to share a link or private copy?
> ask the people who were there
I've attempted private exchanges about this stuff with some folk who were there. The general response is a reluctance to talk about it. What little I get beyond that doesn't seem consistent with the picture you paint.
> You're spreading misinformation
If anything I said is wrong, please point out the specific issues. Thanks.
Many folk have contributed to both projects. But yes, they were (and remain) separate projects, with Patrick at that time being the point person for Rakudo, not Parrot, and others being point people for Parrot, not Rakudo.
> {the teams running Rakudo and Parrot} had a limited understanding of each others' perspective
Perhaps.
My own understanding (which is admittedly also merely armchair analysis after the fact for most of the period 2006 thru mid 2011) goes something like this.
First, the gap between Rakudo's purpose and Parrot's purpose became (or maybe was from the get go) an irreconcilable conflict.
(Rakudo's purpose is to ship a practical P6 compiler, including a multi backend strategy to address Larry's "must target VMs" requirement. Parrot's purpose is to ship a practical VM for dynamic languages, with P6/Rakudo given no special status.)
Probably the last chance to have gotten things back on track, which I define as Parrot trying to serve P6's and hence Rakudo's needs, even if it did other things, was 2010. Patrick was made a member of the Parrot Foundation's small board in 2009. Rakudo Star was coming. jnthn had designed and implemented 6model and NQP and there was a chance Patrick could have convinced Parroteers to replace the broken stuff in Parrot with these pieces.
But then, just as Patrick needed to be most available to steer Rakudo (and perhaps to steer Parrot too) through the very difficult year 2010 (to produce, ship and support an initial Rakudo Star; to evolve Rakudo in the face of the problems of fit with Parrot; to try keep Parrot viable), he suddenly had to drastically lessen his involvement -- falling to near zero in late 2011 and since.
(NB. Please don't say this was Patrick exhibiting the "leave phase" syndrome you talked about.)
At this point, instead of the previous situation in which the conflicting directions were more implicit than explicit, it now became clear Rakudo and Parrot were going to do their own things. They might both survive and work with each other. They might not.
In early 2011, commits to the master branch of the Parrot repo fell of a cliff.
By 2012 it really looked bad for Parrot, judging by the paucity of commits and #parrot (in)activity.
In spring 2012 jnthn secretly started MoarVM. He reasoned that if by some miracle Parrot got back on track in a way that obviated the need for MoarVM he could mothball it or transfer the best pieces. If not, he'd charge right on ahead and finish MoarVM.
Fast forward to 2014. Reini Urban is about the only person working on Parrot and it is currently falling further behind in supporting Rakudo's current needs. For example, concurrency. In contrast MoarVM now far outperforms Parrot as a backend for NQP and Rakudo and looks set to pull much further ahead this year.
Multiple Parrot developers volunteered to do this. Rakudo developers said "No", or in some cases "Not yet." Said developers tired of being called out for not supporting Rakudo on one hand and being told not to improve Parrot on the other hand and stopped working on Parrot altogether. You can easily verify this by looking at mailing list archives from that period.
Given I was there when Orwant threw his mug, I thought it would be finished. So the two statements you made seem incongruent to me.
There's only one P6 compiler relevant to most folk: Rakudo.[1]
Rakudo doesn't have a CLR backend.[2]
Parrot was supposed to be purpose built for the P6 project which in turn was supposed to be implementing Larry's specification. In practice Parrot grew its own purpose which ended up increasingly conflicting with Rakudo's to the point of irretrievable breakdown in 2009 (even if that wasn't obvious at the time). In contrast, MoarVM is purpose built for NQP and thus Rakudo and thus P6.[3]
----
[1] Rakudo was started by Patrick Michaud, the Perl 6 project manager, in 2004. It was always supposed to produce a reference P6 compiler implementation. All other compilers, including Pugs and Niecza, were projects started by enthusiastic individuals without a mandate to produce an end result. These latter have all ended up stalling far short of completion.
[2] Rakudo does not run on the CLR. jnthn, the leading P6 dev who ported Rakudo to JVM and MoarVM, prototyped 6model (a key piece of any port of Rakudo) on CLR, and teaches C# for a living, has suggested that a port to CLR will be just a SMOP. But he has also made it clear he's not planning on doing the port himself.
[3] Parrot is a very poor performer as an NQP/Rakudo backend (and is falling further behind as NQP and Rakudo evolve and Parrot fails to keep up). In addition to speed and RAM issues, if you want to use P6's concurrency features today, you must use either the JVM or MoarVM backend.
Stable releases of Perl 6 and Gnu Hurd -- like commercial fusion power -- are permanently just around the corner...
So, not really a JVM bug, but rather a bug in Rakudo for the JVM. I'd fix the readme, thanks.