Would be perfect if it supports an external database and not only sqlite.
222 karma · joined July 8, 2014
Would be perfect if it supports an external database and not only sqlite.
The old web (html and links) work fine with tools like Scrapy, but for modern applications which rely on javascript this does no longer work.
For my last project I used a chrome plugin which controlled the browsers url locations and clicks. Results where transmitted to a backend server. New jobs (clicks, change urls) where retrieved from the server.
This worked fine but required some effort to implement. Is there an open source solution which is as helpful as Scrapy but solves the issues provided by modern javascript websites/applications?
With tools like Chrome headless this should now be possible, right?
Are you going to use this as is in production, or do you have an internal version it?
It would be really cool if this could be controlled with two apps. One on a smartphone in the box, one as a cardboard monitor. Or is it already like that?
I can't imagine what the numbers are. Is there any big player which has public numbers about their infrastructure? (I mean in detail - which services, how many instances, traffic, ...)
Btw. using an orchestration platform simplifies many of those aspects of "one node is going rough and I've to do accidentally something stupid".
Part 1: create snapshots
https://serverlesscode.com/post/lambda-schedule-ebs-snapshot...
Part 2: expire old snapshots
https://serverlesscode.com/post/lambda-schedule-ebs-snapshot...
If not what other local deployment method do you suggest that works good? (All other methods seem to be deprecated)
So there is no "serverless" which is only spinning up the container as needed. Or scaling up the containers if required. Hoped to see something like that. :)