Hacker News Reader

Comment by saryant | Hacker News Reader

The chart says Play Framework didn't complete but looking at the output, the logs say it did.

https://github.com/TechEmpower/TFB-Round-10/blob/master/peak...

What am I missing?

An error occurs, which is logged to stderr, but the benchmark logs don't capture stderr so it's hard to know what's happening. (Or maybe Play redirects stderr to a log file?)

The test passes in the preview runs, in the TechEmpower continuous integration tests and in the EC2 tests so it's probably some transient error that only occurred in the final bare metal test. Maybe there's a race condition in the Play 2 test scripts which only shows up sometimes.

I've spent a fair bit of time maintaining the Play 2 benchmark tests so it's very frustrating to get no result on the final test. Oh well!

Out of the box, the start script from "play stage" does not redirect stderr.

Though I didn't think to check the classpath when I was poking around the TechEmpower github repo. I wonder if a logback.xml slipped in somewhere that's siphoning off stderr to some unknown destination?

It's not that they didn't complete, they never even started. Definitely configured incorrectly for the tests.

Agreed. Our logging has undergone some solid improvements in the last week or two, and so round 11 will, if not resolve this issue completely, make the logged output more useful for tracking down issues like this during the preview runs.

Sorry, the links to logs were going to logs from a preview run. I've just changed the links to direct to the final logs. In the final run, play2-scala did not respond:

https://github.com/TechEmpower/TFB-Round-10/blob/master/peak...

Generally that error happens when Play tries to bind to a port that's already in use. Looking at the start scripts for the Scala projects, the RUNNING_PID file is just being removed if it exists but your script should probably kill that PID if the file exists before deleting the file.

https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

Our suite takes a nuke-from-orbit approach when it comes to killing processes, as this has come up in every round. The idea is that now all tests are run with a specific user, and instead of relying on the test to shut down properly (which MANY could not reliably do), we simply nuke all processes owned by this runner.

It has the downside that if a process forks other processes and drops them into another different user (recently addressed for hhvm, for example), we cannot capture that. However, we have made great strides in trying to avoid that. Additionally, the logging for the application WOULD suggest if a port were bound prior to start-up, and that does not seem to be the case in this example.

Yeah, you should see a java.net.BindException in that case. Play sends it to STDERR rather than STDOUT, I thought maybe that output was being redirected but that doesn't appear to be the case.

The other primary cause of that "oops" message is when evolutions can't be applied but that also doesn't appear to be the case.

Maybe they need to pass `-Dhttp.address=...` to the start script[1]; Play binds to `0.0.0.0` and port 9000 by default.

Here's the corresponding error from the log in Play[2]. Not sure what else it could be...

[1] https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

[2] https://github.com/playframework/playframework/blob/2.2.x/fr...

Reply on news.ycombinator.com