164 karma · joined July 24, 2011
from sys import argv
from unidecode import unidecode
from newspaper import Article
import re
script, arturl = argv
url = arturl
article=Article(url)
article.download()
article.parse()
title2 = unidecode(article.title)
fname2 = title2.lower()
fname2 = re.sub(r"[^\w\s]", '', fname2)
fname2 = re.sub(r"\s+", '-', fname2)
text2 = unidecode(article.text)
text2 = re.sub(r'\n\s*\n', '\n\n', text2)
f = open( '~/Desktop/' + str(fname2) + '.txt', 'w' )
f.write( str(title2) + '\n\n' )
f.write( str(text2) + '\n' )
f.close()
I execute via from shell: #!/bin/bash
/usr/local/opt/python3/Frameworks/Python.framework/Versions/3.7/bin/python3 ~/bin/url2txt.py $1
If I want to run it on all the URLs in a text file: #!/bin/bash
while IFS='' read -r l || [ -n "$l" ]; do
~/bin/u2t "$l"
done < $1
I'm sure most of the coders here are wincing at one or multiple mistakes or badly formatted items I've done here, but I'm open to feedback ..."DOCTYPE HTML / if lt IE 7 / html class no js IE6 oldie!"
That polka lyric jams.
https://threadreaderapp.com/thread/1135354815259656192.html
And a bookmarklet:
TLD -> domain -> subdomain -> string
For example, www.metafilter.com and ask.metafilter.com would be sorted close to one another, as opposed to the former being in the "w"s and the latter being in the "a"s.
Really wish that had become the standard ...
http://www.cliffc.org/blog/2017/07/30/introverts-emotional-p...
I've been a fan of Chiang for a very long time. I'm a real fan of superintelligence fiction [2] and "Understand" [3] was the first to catch my attention.
[1] http://dresdencodak.com/wp-content/uploads/2009/09/2009-09-2...
[2] https://ask.metafilter.com/286008/Seeking-Hyperbrain-Books-F...
[3] https://web.archive.org/web/20140527121332/http://www.infini...
Dems said in order to participate in debate, you must poll 1% in three polls IN the six weeks prior to the debate. Depending on what polls were counted, Lessig either qualified or were one poll away from doing so.
Dems then “clarified” rules as being 1% in three polls AT LEAST six weeks prior to the debate.
HuffPo op-ed from Lessig's campaign manager: http://www.huffingtonpost.com/steve-jarding/the-democrats-ha...