HNHacker News
TopNewBestAskShowJobs

yago

10 karma · joined July 15, 2008

submissionscomments
yago··on The Easy Way to Extract Useful Text from Arbitrary HTML
from TFA: The concept is rather simple: use information about the density of text vs. HTML code to work out if a line of text is worth outputting. (This isn’t a novel idea, but it works!) The basic process works as follows:

   1. Parse the HTML code and keep track of the number of bytes processed.
   2. Store the text output on a per-line, or per-paragraph basis.
   3. Associate with each text line the number of bytes of HTML required to describe it.
   4. Compute the text density of each line by calculating the ratio of text to bytes.
   5. Then decide if the line is part of the content by using a neural network.
You can get pretty good results just by checking if the line’s density is above a fixed threshold (or the average), but the system makes fewer mistakes if you use machine learning — not to mention that it’s easier to implement!

for whose who too scared to follow link...

yago··on Rate/Review my startup - mixturtle
looks awesome. two questions: 1) which technologies was used? 2) how do you handle high loads?
yago··on Pandora for iPhone will be a huge hit
One of empirical data mining law in action: quantity of information beats quality.