Finding rigorous engineers is hard.
1,139 karma · joined May 6, 2009
https://twitter.com/haxor
Finding rigorous engineers is hard.
https://plus.google.com/u/0/116969159384245484847/posts/Rb44...
Edit: Anyways, if you want to repurpose this thread for constructive criticism that's fine. I'd rather this site fixed things instead of searching for a replacement.
"Hacker News has gone to shit; help me find a good editor for a replacement"
https://twitter.com/haxor/status/225088232721879040
Just another example of how things are done around here
http://sparknlaunch.wordpress.com/2012/04/05/project-mackere...
"The programmer has a huge advantage now because he has delayed his competition by convincing them to learn an unfamiliar skill before breaking ground."
Love it.
Write code. We have Go, Java, JavaScript, Python, Perl in the repo. Multiple implementations are the goal.
On the other hand, the Pipeline API is task oriented. Developers use it with a procedural approach. The focus is on parameter and return value passing and scheduling. It's easy to reuse your existing code in this framework. Think of it as something closer to a parallelizable Bash than a data processing framework.
Say you wanted to generate a heatmap using MapReduces. How would you do it? You'd probably need something like this:
1. Map location data points to (region -> weight)
2. Reduce (region -> weight) to (region -> sum of weights)
3. Map data points to (region -> 1)
4. Reduce (region -> 1) to (region -> sum of points in region)
5. Shuffle output of #2 and #4
6. Reduce (region -> sum of points) and (region -> sum of weights) to (region -> average weight)
The pipeline API makes it easy to describe the dependencies between these separate MR jobs, wait for each segment to complete before triggering the next, and lets you reuse this logic as part of a larger computational workflow.The Mapper framework/MapReduce integration part is not ready yet, but we're getting there. Release early/often~
ps. For those of you who know how to do a heatmap in a single MR: I'm just trying to demonstrate why you may need to pass inputs/outputs between MR jobs.
""The second new feature is that we've enabled 'wildcard' domains for App Engine apps serving off appspot.com. What this means is that you can now create multiple subdomains for your App Engine app, and have them all served by the same application. Thus, your users can access 'myapp.appspot.com', or 'developer.myapp.appspot.com', or 'news.myapp.appspot.com', with your app deciding how to handle each request. No setup is required to use the wildcard domains - simply configure your app to serve up requests to these subdomains however you wish. You can detect which domain a user requested by looking at the 'Host' header in the incoming request - for example, in Python's webapp framework you can access this with self.request.headers['Host'].""
I think this is solvable. One option: