zeratul

That's great news. Keep doing it. I was thinking about using finite state machines to generate eye-catching structures. I'm guessing that you are using genetic algorithms to create populations of lurepets and improve them using users' input.

You could build a google app engine web app to do that. Users that will train your genetic algorithms will get discount for their t-shirts. You would use google "in-app" purchases API to sell your t-shirts. If you get enough data from users you can try to personalize, e.g. association rules, if someone bought t-shirt A then he is likely to by t-shirt B.

Note to self: web app that uses genetic algorithms to improve its product and association rules to sell more is also a data mining web app.

I like the idea of generating an image and selling it as a t-shirt but you have to do better than this.

Website design is not very appealing. Have a look what other people do: http://dribbble.com/

Your LurePets are not a new idea. Zazzle is selling them already: http://www.zazzle.com/lurepets+gifts . Personally I like fractal t-shirts more: http://www.zazzle.com/fractal+tshirts

But don't give up. It's a good idea that needs better front-end and more advanced math and pattern generation at the back-end. If you auto-generate images that no one else has and are appealing then you might have a sustainable business.

I'm still trying to figure out what people mean by "big data". I'm full of hope that they don't mean "big storage" or "big hard drives filled with big bytes".

The data by it self does not have any value. Maybe some need to store data just for legal purposes. If ever sued they can show the records. The value comes from summarizing the data into something that you can use it for making decisions. So my hope is that media are really saying: "there will be a lot of data to analyze - start practicing now."

Thumbs up for sense of humor. There is little bit about kernels on http://www.ml-class.org

I'm looking for a good implementation of Relevance Vector Machines. One that I could port to R or Python 2.7 (NumPy). I found two interesting C++ libraries and posted them on HN.

Note to self: data mining web app; few visual bugs; duplicates; I can see only "Bad News! This Item Already Sold."; no info where an ad comes from; they use jquery; actually http://www.shopobot.com/ by nostromo looks better (but they just focuse on amazon price monitoring);

Nowadays the graphics cards are so fast that neither games nor APIs can keep up.

No really, when we will get something as simple as OpenMP to do our data mining on GPUs? There is more data to be processed than there are games to play.

gavanwoolery: What you describe is the Renaissance. Dark Ages could be DOS or WIN 3.1 with PASCAL or VB and its short dominance over classical and forbidden beauty of Unix and C. I guess the title of the article should really read: "The Renaissance of Developers".

Indeed, we are like the Renaissance men. We read a lot, learn a lot, and write a lot; different languages, different platforms, and different standards. We need to know how to admin, design, write, build, test, and deploy. Each of us has to be Galileo, da Vinci, Columbus, and Kepler in one person.

What we are waiting is the era of positivism where methodological debates concerning clarity, replicability, reliability, and validity established modern science governed by experimentation and sound statistics. Mathematics applied in Physics, Physics in Chemistry, and Chemistry in Biology bring order and sanity the same one that developers are longing for in computer science and computer engineering.

bedatadriven: R native code is usually slow and always memory hungry. Nonetheless, running R on Google AppEngine is very tempting. Could you give us some idea how the memory usage looks like using Renjin when compared to any other R distribution? Here is example how to measure memory:

http://heuristically.wordpress.com/2010/01/04/r-memory-usage...

Note to self: HP (OVIS-2) and IBM (Watson) are working on "intelligent" load balancing tools; is real-time cluster analysis possible? single-linkage (MST cluster cut)? would that be useful? balancing physical nodes with virtual, seems like some sort of optimization?

edw519: I know it's just for fun ... but on two occasions I've heard that I.T./I.S. bosses are purposely chosen NOT to be computer savvy because the otherwise operations cost too much. CEOs want good enough not perfect. This might create interesting team dynamics or sunk a project. I wander if there is a number which happens more often?

Very interesting. Are you into chaos theory? Some day I would like to investigate if behavior on HN, say average score on the front page over time, lies on a strange attractor or not.

Congrats!

Note to self: great visualization; very good javascript; what would I do to get those guys on my team ...

Note to bastian: you may consider adding viewport statistics to your google analytics account (you can add many other useful events the same way):

http://wishfulcode.com/2011/11/02/measuring-viewport-size-wi...

That might be true for some people but not everyone. Do you take into account situation where someone can get treatment and change ones mind? Since you, a non-medical professional, cannot make that judgment it's better to use as a rule to always call.

Emergency visits are not about locking someone away but about figuring out a best treatment. Whether a person will follow the treatment or not that's a different story but at least someone was there for them, to listen, to help, and with means and experience to do it.

I can only guess that jey's comment was downvoted because R is more prevalent language for statistics and data mining when compared to Python:

http://www.kdnuggets.com/polls/2011/languages-for-data-minin...

Thus, there is a much larger ecosystem as it comes to STATISTICS and data mining in R. Cluster analysis packages are especially advanced in R.

BTW, there is an effort to have OpenMP support in core R, e.g. the "dist" function:

http://r.789695.n4.nabble.com/How-to-safely-use-OpenMP-pragm...

EDIT: I was able to compile and run some of the functions under MacOSX 10.6 and R 2.13.1. I couldn't get it to compile with CULA (free version) and my GPU hardware is 1.1 so I can't run SVM :-(

Let's not forget about the civic duty to call 911 if you suspect that your friend or your neighbor is suicidal. Emergency medicine has experts that can help much more than National Suicide Prevention Lifeline.

Are all entires user contributions? Is it be possible to scour Internet for these kinds of events? Twitter?

Some old stories show number of comments while some don't. Could that be related?

My data shows that the stories on the front page had unusual high scores last night.

Ones, that I would focus on:

Model selection http://www.ml-class.org/course/video/embed?video_id=62&s...

Diagnosing bias vs. variance http://www.ml-class.org/course/video/embed?video_id=63&s...

SVM Optimization Objective http://www.ml-class.org/course/video/embed?video_id=72&s...

Large Margin Intuition http://www.ml-class.org/course/video/embed?video_id=73&s...

Mathematics Behind Large Margin Classification (Obligatory!!!) http://www.ml-class.org/course/video/embed?video_id=74&s...

Kernels I http://www.ml-class.org/course/video/embed?video_id=75&s...

Kernels II http://www.ml-class.org/course/video/embed?video_id=76&s...

Using An SVM http://www.ml-class.org/course/video/embed?video_id=77&s...

... yep, you don't need to login.

IBM Watson team is focused on medical data mining now. Training IBM Watson just for Jeopardy took over 7 years. They are struggling how to make Watson learn faster. It's not the CPU or RAM limitations. There are a lot of custom pipeline building that needs to happen. Medical data is usually behind firewalls, password protected. Even medical books and articles are not easily accessible. There is no place for side projects.

Good exercise.

Note to self: they got the data; they didn't add much value; they presented the data with fairly clean design; it could be argued that this is a data mining web app but there is a flaw if the plan is to produce income by engaging the user.

Note to feldmanr: http://www.programmableweb.com/ says that there is 4535 APIs. It's not even possible to read the documentation for a subset of them to figure out whether you can get an incremental user ID or not. But even if you did and even if you used some of them you are just producing a fun fact. Hardly a reason to come back for more. Yet, the exercise is a worth while; one gains knowledge about heterogeneous nature of web APIs - a skill necessary to build profitable data mining web apps. Please, do come back with an idea that can clearly retain your users beyond single visit.

Very nice!

Note to self: they got the data; they added a value; they presented it nicely and neatly. Good data mining business model.

Note to jack7890: as it comes to events, there are two factors that play a role here: time (you got that in the calendar view) and place (just partially in the "theatre view"). I have two ideas for improvement. How about showing events in the calendar and color code the distance (e.g. red close, blue far away)? Alternatively, you could show a 2D map, user in the center, events spread on the map and color code the time (e.g. red - soon, blue - far in the future). You can have user put zip code or get it by IP (might be not accurate for mobile devices, though).

I just use it offline, on my laptop. The UI is just a nice frontend to an idea that you know will work. Then you might have a ton of ideas that you want to check before publishing. One question may want to answer is if the behavior on HN predictable?

Google App Engine doesn't support R but you can use NumPy:

http://code.google.com/appengine/docs/python/python27/using2...

http://numpy.scipy.org/

The content is refreshed every 15 min - both database and UI.

Probably the production version of the code will get memcache and some other features. Apart of couple suggestions from the discussion here the educational version of hnpickup will be frozen. Although, any simplifications or correction to the code or documentation are very welcome.

Just got an email from Kontera ads:

  Denial Reason: Insufficient Text.

What happen to saying: "picture worth thousand words"?

Pickup ratio suppose to measure your chance of getting from "newest" page to the "news" page. But the more I think about it the more I'm inclined to think it's just a very hot topic detection device.

Pickup ratio > 1 means that the "newest" page has couple very interesting stories that are also seen on the "news" page.

Pickup ratio > 2 means that there is some sort of hysteria on HN about some particular topic ...

Yes!!! I was hoping to have pay-per-impression advertisement that would pay the bill but apps which don't offer wordy clutter are not allowed to have ads. I was rejected by adSense, BuySellAds, and adBrite. Sometimes I think we build Internet for robots not for humans.

I have ~100 visits per day. That costs me ~$0.10 per day.

More details here:

https://github.com/entaroadun/hnpickup/wiki/Hacker-News-Pick...

Thank you.

btbytes did the md formatting for me. I will accept the pull request as soon as I get home.

If you click on the question mark in the top right corner you will get the definitions. More documentation is in the source code.

EDIT: As I say in the "warning" section of the readme file: usually it's not a good idea to show the raw data because it will confuse the end user. And if you add more explanation you will cause clutter and cognitive overload. I show the raw data mostly for people that want to dig into the source code.

Poll results roughly correlate with my web app visitors ("When is good time to post on HN?"):

  California + New York 2,516

  Rest of U.S. 3,661

  United Kingdom 797 

  Canada 618 

  India 364 

  Germany 293 

  France 233 

  Australia 221 

  Netherlands 133 

  Sweden 131 

  Italy 96

Thanks. My github handle is entaroadun. The iframe's html line 10 says src="/static/button.js" while it should be src="/static/js/button.js"

BTW, there is an odd feature on HN: it's possible to post same URL after ~19 days as new story. This will cause unintended consequences for all HN buttons. While investigating this issue I found that one can upvote his own story by re-posting it. I guess some variable gets removed/reset on HN servers I just couldn't figure out what but I'm couple hacks behind.