[0]: https://github.com/peterldowns/bookshrink [1]: http://www.bookshrink.com/
Don't be so hard on yourself. I review papers for CS conferences and just the fact that you used TF-IDF weighting puts you well above the average.
""" SPLIT INPUT INTO SENTENCES"""
god_awful_regex = r'''(?<!\d)(?<![A-Z]\.)(?<!\.[a-z]\.)(?<!\.\.\.)(?<!etc\.)(?<![Mm]r\.)(?<![Pp]rof\.)(?<![Dd]r\.)(?<![Mm]rs\.)(?<![Mm]s\.)(?<![Mm]z\.)(?<![Mm]me\.)(?:(?<=[.!?])|(?<=[.!?]['"]))[\s]+?(?=[\S])'''
Be advised that one of the nice things about python reg-exes is that they allow in-line comments (and naming of groups), if compiled with the verbose-flag:
""" SPLIT INPUT INTO SENTENCES"""
verbose_regex = r'''(?<!\d) # I can't actually tell
(?<![A-Z]\.)(?<!\.[a-z]\.) # what you're doing here...
(?<!\.\.\.)(?<!etc\.) # Is this one big group, or is
(?<![Mm]r\.)(?<![Pp]rof\.) # it several groups, with
(?<![Dd]r\.)(?<![Mm]rs\.) # different prefixes?
(?<![Mm]s\.)(?<![Mm]z\.)(?<![Mm]me\.) # Clearly, it's got something to do
(?:(?<=[.!?])|(?<=[.!?]['"]))[\s]+?(?=[\S])''' # with
# not matching the dot at the end of Dr.
# or as part of an ellipsis as the end of a
# sentence? But my point was that if such a
# regex is built-up and tested with comments
# it can be quite readable
god_awful_regex = re.compile(verbose_regex,
re.VERBOSE)
# continue here...
[0]: https://github.com/bumptech/tl-dr.js/blob/master/tldr.js