kmavm

Yes, exactly. Thanks for mentioning it.

The aspects of PHP that are bad (and there are more than a few) are easily tooled around. We have a review-time tool that parses your PHP, symbolically executes it, and looks for dubious idioms, flagging them for you and your reviewer.

Meanwhile, PHP gets a number of things very right. For instance:

* State: there is none across requests. Oh sure, there's APC, and the filesystem, and databases, etc., but those are all accessed via libraries. There is no PHP-level state that survives a request, which provides natural fault isolation: a request which goes wrong doesn't take the server down with it, or live on to corrupt other requests.

* Concurrency: PHP's concurrency model is the web request. If you want to run some code in parallel, curl to localhost. If you wrap an appropriate library around this, it looks just like Erlang's actors. This is shared-nothing concurrency, which turns out to be a compelling local maximum.

* Interactivity: The development workflow PHP provides is very productive. You save and load the page. You don't save, optionally recompile, restart the server, wait for it to initialize, forget what's different from the last three times you tried this because this has all taken three minutes and you forgot to write it down, etc.

These three things turn out to be among the most important parts of a server-side language, and PHP gets them very, very right. The rightness was accidental, not by design, but this doesn't make it any less right.

Also, the language itself can be improved. In the HipHop toolchain, -v Eval.EnableHipHopSyntax=true turns on a bunch of (backwards-compatible) features we've added to our PHP dialect, the most notable of which are XHP (syntax for using XML literals as PHP expressions) and generators (inspired by Python's, using similar 'yield' syntax). I put up some sample code for a talk just the other day, if you're curious: https://github.com/kmafb/hiphop-samples

Hey. I work on Facebook's PHP toolchain, HipHop, which these days includes a debugger, interpreter, JIT compiler, and the ahead-of-time compiler. The fact that the AoT compiler uses C++ as a target doesn't change the input language, which really is PHP. E.g., $i = 0; $a[$i] = $i++; makes a different array than $i = 0; $a[$i + 0] = $i++;. We have to get all the nooks and crannies right because we use significant open source packages in PHP.

The developer's usual routine is the same as someone using Zend; they write honest-to-goodness PHP and reload the page. There is no compilation involved; you instead are using the JIT (or until recently, an interpreter) that monitors the filesystem for changes and silently integrates them.

Facebook devs use PHP exactly as everybody else does; they just do so with a different toolchain.

Please don't mistake me: PHP-the-language is not well-designed. And we absolutely would never have gone down this road if we didn't have a metric ton of PHP to run and continue developing.

There are a few things PHP gets accidentally right, though, and one of them is the execution model. Each request starts with a brand new empty PHP environment; runs in a single thread to completion; and must keep persistent state off to the side in a database or memcache or what have you rather than in language-level objects. This turns out to have been a really good set of decisions. It lends itself to a nice tight development loop: instead of save/recompile/restart the server/test, you basically save/load the page. It's easier to remember what the heck you were doing when you get feedback on the code you wrote in 500 milliseconds.

I am 35 and have a wife and family. And it is not my first time at the tech company rodeo either.

Because I find it fun. If you don't, that's perfectly cool, too.

Hey, my name is Keith Adams, and I'm an engineer at Facebook.

I work on the HipHop virtual machine, a dynamic compiler and runtime for the PHP language. It is the hardest challenge I've tackled professionally, and the people I'm working with are brilliant and work like animals.*

I think the larger significance of our work on HipHop is in the context of software as a whole. PHP is one of those "developer productivity languages," like Python, JavaScript, LUA, etc. And, though I was skeptical before I came here, developers really are more productive in these languages. It would blow your mind what world-class people working in this medium can accomplish in compressed timeframes. This means that increasing these languages' performance is incredibly leveraged; to the extent we're successful at making PHP faster, we change the set of problems that PHP can address. Moving a problem from the "must be solved in C++" category to the "can be solved in PHP" category frees up our fellow developers to get more done with their finite professional lives.

*Edit: By "like animals", I mean, "with a survival-level sense of urgency." It doesn't necessarily mean long hours.

> there are more automobiles worldwide than there are Facebook users.

http://www.wolframalpha.com/input/?i=how+many+cars+are+on+th...

The HipHop compiler will optimize much, much more effectively if non-trivial eval() is disallowed, as it is in FB's production code. Obviously, since it's an ahead-of-time system, complex code in eval() will run slowly. The HipHop virtual machine is perfectly happy with eval().

Which canonical PHP.net implementation? They differ across minor revisions, in intentional and unintentional ways.

In the absence of a standard, saying what is and is not PHP is necessarily a practical matter: useful PHP implementations are those that run non-trivial PHP applications. HipHop qualifies.

FB's dependence on HipHop is because of backwards-compatible extensions to the language (like yield, e.g.). These extensions don't prevent HipHop from running normal PHP programs, though our use of them does prevent us from using Zend. The extensions are under flags that default off, if you don't want to use them.

"Prosaic license"? If the essay was written in iambic pentameter, it certainly escaped me. Anyway, "C++ macro language with striking similarities" has a pretty precise technical meaning, which sane, literate people might actually mistake.

1) Great question! Yes, it is super-common. Most of our type inferences on local variables are actually of form "Null-or-X". So we can emit code that skeptically checks for Null, but then goes ahead and assumes X otherwise.

2) I'm not claiming we have all the answers here; but decent compile times has been a strong evolutionary pressure all along.

3) Yeah, we basically implement the "slightly relaxed" semantics described in that paper, and are addicted to the performance wins from doing so. WRT loosening semantics, though, in general you'd be surprised how close we come to quirk-for-quirk compatibility. The problem is that real PHP programs, including ours, end up depending, often unintentionally, on those quirks. I.e., nobody meant to rely on the order function params are evaluated in, but in practice we rely on it because param 0's evaluation has some side effect that param 1's evaluation needs to run correctly...

"(And strictly speaking, Facebook isn’t written in PHP; it’s written in a C++ macro language with a striking resemblance.)"

I'm a Facebook engineer who works on the HipHop compiler and HipHop virtual machine. It's in PHP, absolutely full stop.

It's amazing how much the fact that g++ is involved somewhere in the toolchain confuses people in this matter. C++ is just an intermediate representation; the source language really is warts-and-all PHP.

I haven't done any work on the type inference stuff, so my understanding is second-hand. My peers who actually know what they're talking about are working on a more complete write-up, so stay tuned.

1. (Type inference.) The compiler's type inference is basically a first-order symbolic execution without inlining; we don't iterate to a fixed point or anything, and any union types at all cause us to give up and call it a variant. Many other PHP surprises mess up type inference; for example, function parameters by reference aren't marked in the caller, so if anywhere in the codebase there exists a unary function that takes its first parameter by reference, $f($a) will spoil our inference of $a: $f might mutate it.

Still, simple things work pretty well. One major area we do well on is inferring the return types of function calls. It's hard to say how much we're missing out by not doing more sophisticated type inference; while we can count the sites at compile time, this doesn't tell us which sites are dynamically important. Our anecdotal experience has been that type inference is one of the more powerful optimizations we can perform.

We don't model refcounts, and I'm certain there are opportunities there.

2. (Compile times.) 15 minutes sounds about right to me, and chuckr would certainly know better. Compiler speed itself is something we pay attention to, since it inhibits our ability to turn pushes around. The actual global AST analysis is multithreaded, so we keep the whole build machine busy. So far the trade-offs between optimizations and compile time have not been too dire; there haven't been things that would make the binary twice as fast if only we could compile twenty times as long, e.g.

3. (PHP implementation/semantics). Who've you been "arguing" against? :). It's a manifest fact that accidents of PHP's implementation have been sunk into the language. Destructors are one of the ways to witness refcounts, but there are others. var_dump() of arrays exposes the refcount of internal data; things whose refcount is >1 are displayed with a '&' preceding them, e.g.:

<? $a[0] = 1; var_dump($a); $data =& $a[0]; var_dump($a); unset($data); var_dump($a); ?>

produces

array(1) { [0]=> int(1) } array(1) { [0]=> &int(1) } array(1) { [0]=> int(1) }

Less trivially, it is hard to efficiently support the copy semantics for arrays without reference counting; other systems have overcome this, but it is less trivial than with refcounting.

The HHVM JIT isn't in (much of) production yet, because it isn't fast enough yet. Once we have high confidence it can provide some perf wins, we'll go ahead and move over to it. Even if it were perf neutral, there is some virtue to having developers use the same environment as production.

This is a long and deep subject.

I wouldn't say that we've "continued on the PHP path." I'd say that we've refused to throw out the precious PHP parts of our application, while not being afraid to use more appropriate languages across Thrift boundaries when needed. Our search engine, newsfeed, and ad serving infrastructure, for instance, are in C++.

A drop-everything-and-rewrite of the PHP code is entirely out of the question, for all the reasons covered in Spolsky's 12-year-old classic on the subject: http://www.joelonsoftware.com/articles/fog0000000069.html. Those of us working on making PHP perform better are a tiny fraction of Facebook engineering as a whole; this small overhead cost is nothing compared to the risks inherent in a ground-up rewrite.

Most of PHP's language-level faults can be engineered around. For instance, we have a code-review-time script that parses (really parses) the code to warn engineers (and reviewers) about dangerous or deprecated idioms.

PHP also has some affirmative virtues. The programming model is more productive than that of compiled languages, and even many interpreted languages; save/reload the web page is just a better, tighter loop to get work done in than save/compile/restart my server/reload the web page. I'm actually a fan of PHP's concurrency model, which naifs often mistake as "no concurrency allowed"; PHP's concurrency primitive is curl[1], and if you wrap a tiny bit of library around it, you can make it behave like actors.

[1] Seriously. curl provides a shared-nothing way to asynchronously run code, and has the virtue of not caring what language the other side is written in to boot.

(I work on the HipHop compiler.) You start by compiling PHP source.

Simple PHP statements take a lot more space in the binary than intuition suggests. E.g.:

if ($a == $b) ...

would seem like it should be

  cmp $rax, $rbx
  jz ...

But! If type inference has failed, we don't know what types $a and $b are, so they might be strings or objects or something crazy. So we're going to have to indirectly dispatch to $a's '==' method. We also spend a ton of space on reference counting code; the semantics of the language basically force you to do naive reference counting, since refcounts can be witnessed in various ways, so every time we pass an argument, do an assignment, sometimes even evaluate expressions, we need to manipulate reference counts, and if they've gone to zero call a destructor.

It ends up making the code really large, and one of the things that's unique about our efforts to run PHP fast relative to other dynamic language efforts is that sheer code bulk ends up being our largest enemy; if we're not careful, icache misses eat us alive.

Finally, I'll note that it's not quite a 1.5GB binary. The actual ELF binary is something like 1.1GB, and the remainder of the package we bittorrent around production is stuff like static resources (javascript, css) and primed contents for the APC cache that we want prepopulated on boot.

WRT Self: some of Agesen's work on Self work for doing type inference on conventional, ahead-of-time compilation, and they got pretty good results this way! More interestingly, the opportunities the AOT compiler found were different than the opportunities the JIT compiler found, so there is some hope that they will combine constructively. See. e.g., http://www.cs.ucla.edu/~palsberg/paper/spe95.pdf

Pretty close. Once we've translated to machine code, the interpreter doesn't do type checks; it blindly enters the translation cache for anything that has a translation, and the guards live in the tracelets themselves. The tracelets directly chain to one another in machine code, so they need embedded guards.

Of course. I have kind of a difficult line to walk here; I would like to be respectful to both the authors of the HPHPc compiler, which is a really impressive achievement, and to the many many efforts past and present at running dynamic languages. LuaJIT, various SmallTalk systems, Self systems, Erlang, HPHPc, the JVM body of work, V8,hardware-level binary translators, ... are all among the shoulders we are standing on.

All that said: I get the impression reading the comments here of an emerging misconception that "everybody knows" the right way to run these languages fast. We emphatically do not know. It is still a research problem. Heck, garbage collection, a proper subset of the problem area we're talking about here, is still pretty wide open. Given the smart people who have given decades of their career to it, we should not expect there to be a single, silver-bullet technique that cleanly maps all these dynamic languages to high-performance machine code.

So we need more than one project in the air. Some of those projects should explore whole-program analysis, like HPHPc and some of Agesen's offline work on Self, and decades of Lisp compilers. Some of them should explore runtime techniques like tracing and inline-caching. Some of them should explore unifying these approaches. We need them all.

HHVM is a follow-on effort to the static compiler. We got started long after HPHPc was in production; I started playing around in early 2010, and three of us (Jason, myself, and Drew Paroski) started earnestly putting in full-time work in summer of 2010.

I made up the term "tracelet." Do not use it when trying to sound intelligent. It roughly means "typed basic block," though the compilation units aren't quite basic blocks, and what we're doing isn't quite tracing, and ... so we thought it would be less confusing to just make up a name. Think "little tiny traces."

The bytecode is stack-based for a couple of different reasons, but it will probably remain stack-based because of compactness. Since instructions' operands are implicit, a lot of opcodes are 1, 3, or 5 bytes long. Facebook's codebase is large enough that its sheer compiled bulk can be problematic.

The translator doesn't turn the stack into SSA, instead turning it into a graph where the nodes are instructions and the edges are their inputs/outputs. You can sort of squint at this system and call it SSA where the edges are the "single assignments."

This might be getting less interesting to the general audience, so we can take it offline if you have any more questions. I'm my initials (Keith Michael Adams) @ fb.com.

The static compiler that preceded HHVM was started in a different time; it was 2008, and the JavaScript wars were just getting started. Conventional wisdom (to anybody who hadn't worked on Self in the late '90's) was that running dynamic languages fast was just kind of a lost cause. Since the static compiler was started earlier, and was inherently a more tractable system, it was done sooner. This is something that is hard for a lot of systems people to accept, but software has time value; even if HHVM ends up being faster than the static compiler, without the two years that HPHPc has been providing 2-4x improvements, Facebook would have been less able to make products. And keep in mind, the static compiler is beating our jit at most benchmarks at this writing; improving on it will take some doing.

The only optimizations we're doing are those that bottom-up type inference makes possible; so, we can generate code that knows that strings are strings, arrays are arrays, ints are ints, doubles are doubles, etc. This doesn't sound like much, but avoiding the double-dispatch on things like $a + $b is significant. We're still using the static compiler's front-end, so all of the things it can do (e.g., CSE, constant-folding, deduplicating identical data, compile-time binding of functions and methods) we hope to "inherit" when running in production mode.

PyPy is a very interesting project, with a beautiful approach. I'd be very, very curious to see what an HHVM interpreter written in RPython could do under PyPy. Unfortunately PHP is not scheme; it is a big language, with many non-orthogonal parts, so it would be a significant effort to answer this question. If I were starting over today, I'd take a much closer look at the PyPy approach, but that is just my opinion.

Hey, I work at Facebook on HHVM.

I actually view Facebook as the opposite of what you're describing. VMware, for instance, really did have a bimodal engineering organization; some pieces were amazing, but many were so-so. That can have its strengths, too, and it has worked out ok for VMware to date, but Facebook has the absolute opposite philosophy. For instance, you don't have a "hiring manager" here, because the selection process is decoupled from the allocation process. This means a uniform bar across groups; e.g., I interview people destined for all different parts of the stack, and I expect the same level of skill for all of them.

All engineers, those who work on HHVM like me included, spend their lives in a run-debug cycle trying to "make this darned thing work," whether "this darned thing" is a language runtime or the new photo uploader. The tighter you can make that loop, the more productive the engineer is, because she has less time to keep all the items she's mentally juggling pinned in volatile short-term memory.

(I work on HHVM.) We've dipped our toes into the waters of changing the language a bit. Setting the Eval.EnableHipHopSyntax and Eval.EnableHipHopExperimentalSyntax flags in HipHop enables a few differences, the most notable of which is a Python-inspired 'yield' syntax for generators.

> VMware was probably based on M Rosenblum's graduate work on SimOS, which was GPL.

I worked on VMware's virtual machine monitor from 2000 to 2009. Mendel's students' work on SimOS had nothing at all to do with VMware, full-stop; SimOS was just a full-system simulator, useful for measurement and educational purposes, which has almost nothing to do with virtualization.

The research system of Mendel's that did influence VMware's founding was Disco[0], a virtual machine monitor for SGI's MIPS machines. But the technique Disco used (trap-and-emulate) was inapplicable on x86, and in fact there was peer-reviewed literature claiming the x86 was impossible to virtualize[1]. VMware's first six years of products were instead based on dynamic binary translation; we incrementally recompiled the privileged x86 guest into unprivileged x86 code, in much the same way modern JVMs incrementally compile Java bytecodes into native machine instructions. Our source base was 100% VMware proprietary, and had to be since no open source project was trying anything remotely related at the time.

[0] http://www-flash.stanford.edu/~bugnion/Disco/sosp-html/ [1] http://dl.acm.org/citation.cfm?id=1251316&dl=ACM&col....

I work on a PHP compiler and runtime for a living, so I'm reasonably familiar with its past, present and future. While Ruby and Python are superior in many ways, these three languages (and Perl and JS) have much more in common than they do separating them: Algol syntax, runtime typing, hash-table like duck-typed objects, automatic memory management, some half-baked (for all but JS) concessions to functional style, etc. If you consider them in a world that includes C++, Haskell, Prolog, x64 assembly, Smalltalk, and Lisp, all of these languages form an incredibly tight cluster in the configuration space.

And granting for the sake of argument that Larry Wall wasn't thinking of ksh, sed, grep and awk were just as AT&T proprietary as ksh was.

"Touchscreen that civilians care about."

"Most innovations within the last 20 years came from OSS developers creating incredible techniques used by Apple/MS/etc..."

Name one.

To take some examples I happen to have first-hand intimacy with: Linux is both original and good, but the original parts are frequently not good, and the good parts are clones of Solaris. Touchscreen happened in iOS first. Modern virtualization happened in VMware's closed-source base first. Modern information retrieval is still a proprietary Google secret.

The most important counter-examples I can think of are languages. The "everything is a hash table" descendants of Perl and javascript (in which I include Python, Ruby, and PHP) would not have been viable as commercial products. That said, even javascript began life as a Netscape proprietary implementation, and Perl is mostly a synthesis of ideas present in ksh.

Just as you saw Google suing startups for naming themselves "<X>oo<Y>le" when they were search-related in any way. And as you see every competently run company, everywhere defend its trademarks.

Trademarks and patents are very different things.

http://www.searchenginejournal.com/google-tells-booble-to-ce...