I am part of the team which runs the OpenStreetMap.org infrastructure, we also get a lot of weird and wonderful requests / emails. .mil, .gov, .edu and GeoTLD variants.
I should really do a write-up sometime.
208 karma · joined August 23, 2013
I should really do a write-up sometime.
Technically we able to block and restrict the scrapers after the initial request from an IP. We've seen 400,000 IPs in the last 24 hours. Each IP only does a few requests. Most are not very good at faking browsers, but they are getting better. (HTTP/1.1 vs HTTP/2, obviously faked headers etc)
The problem has been going on for over a year now. It isn't going away. We need journalists and others to help us push back.
The scrapers try hard to make themselves look like valid browsers, sending requests via residential IP addresses (400,000+ IPs at last count).
I reached out to journalists because despite strong technical measures, the abuse will not go away on its own.
Without OSUOSL OpenStreetMap would be more difficult to host and significantly slower to access from North America.
I hope OSUOSL can get the financial support they rightfully deserve.
There are still issues to fix as it is still only a technical preview.
The raster tiles are primarily for our OSM mappers to see their map changes rendered quickly to improve the feedback loop.
OpenStreetMap data is free to download. We published minutely on https://planet.openstreetmap.org/ and the data available via AWS S3 + torrent.
If you just starting out, best to start with a smaller regional extract: https://wiki.openstreetmap.org/wiki/Planet.osm
Our S3 bucket is thankfully supported by the AWS Open Data Sponsorship Program.
rasdaemon also attempts to report which physical DIMM / slot triggered the ECC error.
A scanner that would have caught the vulnerability: https://tech.ovoenergy.com/how-we-prevented-subdomain-takeov...
Or a grey hat scanner for finding sub-domains vulnerable to takeover: https://github.com/m4ll0k/takeover
Because the map "looks complete" Pretoria hasn't really attracted many mappers since his departure.
Cape Town has a much more vibrant community of mappers. I've also been able to get crazy high resolution aerial imagery for CT from the municipality which helps: http://coct.aerial.openstreetmap.org.za/
But the 10% or 3% limit respectively is per TAG (handle). HostEurope have multiple tags (HOSTEUROPE, 123-REG, WEBFUSION, WEBFUSION2, DOTUKDOMAINS, ... )
Nominet (who run the .uk ccTLD) allocate votes proportionally to registrars based on how many domains they have registered. [1]
GoDaddy will now likely control the majority of the votes.
1: http://nominet-prod.s3.amazonaws.com/wp-content/uploads/2015...
Example of an OpenStack vendor guide recommending turning the offloading off: http://clouddoc.stratus.com/1.5.1.0/en-us/Content/Help/P03_S...
Downside is you may have slightly higher CPU usage and/or slightly lower network throughput, I personally have not noticed a significant drop in performance.
udev rule something like: SUBSYSTEM=="net", ACTION=="add", ATTRS{vendor}=="0x8086", ATTRS{device}=="0x10c9", ATTRS{subsystem_vendor}=="0x103c", ATTRS{subsystem_device}=="0x323f", RUN+="/sbin/ethtool -K $name gso off tso off sg off gro off"
Amazon.co.uk: https://www.amazon.co.uk/HP-ENVY-Printer-Instant-Ready/dp/B0... Amazon.com: https://www.amazon.com/HP-F0V69A-B1H-Wireless-Printing/dp/B0...
Section "3. The information we collect" then "5.iv. To third parties from whom you have chosen to receive marketing information."...
Which means they can give your browsing history, numbers you call etc to any company who you have agreed to receive marketing information from. Eg: Tesco Club Card, that-random-website-you-forgot-to-untick-the-box-on.com etc.