1,059 karma · joined October 9, 2011
For example processing 100GB of data actually don't have to be O(nsqrt(n)) because if you process it on cluster, then other machines are also using L1, L2, L3 caches and RAM. Then the whole process can be streamlined which means that some operations can be faster than the pessimistic nsqrt(n).
An example from today: we have a very slow pattern matching code, that starts to be a bottleneck for the application. What can you consider? Well you can dive into sexy bloom filters, experiment with some Trie-based structures. But then when you analyse the problem it results that simple word lookup with a simple hashtable is the fastest solution for given constraints. No big deal, no rocket science.
Probably the same goes for rocket scientists, but one level higher ;)
I was like (。◕口◕。)!!!
What a creepy story
It is true that configure scripts are probably doing some useless things, "31,085 lines of configure for libtool still check if <sys/stat.h> and <stdlib.h> exist, even though the Unixen, which lacked them, had neither sufficient memory to execute libtool nor disks big enough for its 16-MB source code", etc. But then what is the alternative? Writing a configure module each time by every programmer who wants to release some software? This is called code reuse and yes, it's not perfect but it saves time. By not reinventing the wheel again and again. By reusing something that is stable and has been there for some time. Probably such thing is generalizing over many architectures, and making useless things, but then again, who cares for some extra 5-10 seconds of the "configure" command, when you are covered for all those strange corner cases that it already handles?
+ eskill has a good set of diff algorithms, char level, word level, etc.
- lacks line wrapping
- a bit sluggish (but it's not that bad)
Would recommend trying.
- work well with longer files and long lines (i don't know why many of them are very slow on larger files)
- have line wrap option (quite unusual it seems)
- have diff ignoring whitespaces (you would be surprised that not all diff guis support this)
- the best would be to use the same algo that "git diff" does
- if not then any other that works on character-level.
Of course you may use the vanilla git diff, which is really great (even outside of git repositories), but it is not as handy as GUI sometimes.
The closest GUI was meld, but it's not perfect either... macvim with some diff plugin was also good, but not perfect.
I'm looking forward to try it on some bigger datasets.
I also wonder if is it possible to use separately trained word vector model for the supervised task?
How does it compare to the state of the art methods that used those datasets? Is it better or comparable?
When I run the code I see something like: 38/100 = 42.0% - 38.0% accuracy, 90.47619% accuracy of rated data.
90% is nice, but it looks to me like a tradeoff between precision and recall, where we just label those samples with the highest probability of pos/neg...