Jake232

I'm always reluctant to these kinds of ideas, of executing code on/within my database server.

I know it's apparently sandboxed, but that didn't work out too well for ElasticSearch recently: https://jordan-wright.github.io/blog/2015/03/08/elasticsearc....

Elasticsearch takes input in JSON format. It's not quite a web service, but could certainly work for your use case.

I think it's interesting that we can now pay almost anybody we have contact with, at almost zero effort and without even having to remove my debit card from my wallet.

You can pay people via gmail now for your email contacts, send money to your personal friends via social media networking. I bought my Uber to work this morning completely frictionless without having to physically use my card.

It's an interesting development of recent years for sure.

I wrote an article on scraping last year which recieved a lot of praise. May be worth a read - http://jakeaustwick.me/python-web-scraping-resource/

I was also hoping for some sample data to play around with. It would be awesome if Stripe could provide this.

Also thought this. Title is ambiguous (albeit unintentional I'm sure).

It drives enough traffic to kill a general web app. If you've just launched and have something like Rails / Django running on a single app server, it really doesn't that much to overload it. Wordpress seems to not handle it well either (unless you're caching).

Seeing the same here, links to an asset on CoinBase.

https://coinbase.com/assets/buttons/donation_large-36ee93618...

Is there any caching that goes on here? I would presume not as the vendors prices could change prices / details at any point.

I'd be interested in seeing this.

Needs more details, a couple of examples that instantly spring to mind:

Do you spread requests out across multiple IP's to avoid bans / rate limiting?

Do you run javascript on the pages?

SEEKING WORK - Remote (London)

I'm primarily a Python programmer, although also proficient in Ruby. I'm interesting in work in regards to web scraping.

I have extensive experience in this field and can provide dozens of references. I can deliver scripts, or just data exports in SQL / CSV formats.

I have also written a very popular blog post on the topic of web scraping also: http://jakeaustwick.me/python-web-scraping-resource/

Drop me an email at jake.austwick AT gmail.com for more information.

Nice practical introduction to none-programmers, but taught in a way that they can apply the skills to do other things instantly. Good job!

DigitalOcean, spend a couple of hours getting a server setup that covers your general stack and then save it as an image. Then in the future you can just boot a new server on that image for $5 p/m.

I wrote this earlier this year, covers quite a few details, it's been featured on HN frontpage a couple of times though so maybe you have already seen it.

http://jakeaustwick.me/python-web-scraping-resource/

Google isn't that difficult to scrape. I've worked on numerous projects doing it and it just takes a little thinking outside the box and a few proxies.

Probably possible to do, however what's up with just using Last.fm's API?

http://www.last.fm/api/show/artist.getSimilar

PhantomJS can be run through Selenium.

I actually cover this (very briefly) here: http://jakeaustwick.me/python-web-scraping-resource/#thesite...

I should dedicate a section to it though, will stick that on my to-do list.

Where is the job located, wages in London are a lot higher than other parts of the country for example.

Previous Discussion: https://news.ycombinator.com/item?id=7799323

The process going on under the hood (the whole disassemble / AST thing) is pretty unpythonic, however I would argue this then gives the developer the advantage of having a more pythonic query interface to the database on a higher level.

Providing the things happening underneath don't break (and they haven't on me yet), and it generates efficient SQL (which is does), then I'll take the hidden complexity for the advantage of the higher level generator syntax.

Author of the article here (although not submitter). Pony actually has a way to log so that you can see every SQL query being ran against your DB. You can activate it with sql_debug(True). I monitored it for my first few Pony projects, although now I've just come to trust it will do the right thing.

You also can supply direct SQL to the DB via Pony, the documentation covers this: http://doc.ponyorm.com/database.html#using-parameters-in-sql...

Here's a comment on the stack overflow answer I linked to in the post:

Very performant: (1) Bytecode decompiling is very fast. (2) Since each query has corresponding code object, this code object can be used as a cache key. Because of this, Pony ORM translates each query only once, whereas Django and SQLAlchemy have to translate the same query again and again. (3) As Pony ORM uses IdentityMap pattern, it caches query results within the same transaction. There is a post (in russian) where author states that Pony ORM turned out to be 1.5-3 times faster than Django and SQLAlchemy even without query result caching: http://www.google.com/translate?hl=en&ie=UTF8&sl=auto&tl=en&...

I'm pretty proficient in a ton of other languages, so I'm hoping I have a pretty good head start and will be able to follow the book, but I'll read Thinking in C++ in the meantime until my order arrives. Thanks for the suggestion!

Just purchased this. I'm hoping it can make up for a year of not attending my c++ lectures, to learn enough for my exam in 1.5weeks.

That makes sense, thanks.

I've always wondered. How do these guys hit google search so much without hitting limitations?

Google is pretty aggressive at banning bots, and I can't imagine Google have given a competitor API access or something like that. Proxies is out of the question for this scale too.

I feel this could be easily monetized, if you don't get a sale you should look into it for sure. iTunes have an affiliate system, and they have an API endpoint for search. You could simply have some links to purchase the latest album of the currently playing artists / just played artists etc.

Would also love an answer to this.