dangoldin

Man - I'm so impressed by Amazon pushing this stuff out and constantly leveraging their existing infrastructure.

This looks to be affiliated with Junglee (http://www.junglee.com) so might mean Amazon is repositioning it as P2P.

We (adtech) use a very similar approach. We're consuming a ton of data through Kafka and then using Secor to store it on S3 as Parquet files. We then use Spark for both aggregations as well as ad-hoc analyses.

One thing that sounds very interesting and worked surprisingly well when I played around with it was Amazon's Athena (https://aws.amazon.com/athena/) which lets you query Parquet data directly without relying on Spark which can get expensive quickly. I wouldn't trust production use cases just yet and it ties you more and more into the AWS ecosystem but might be worth exploring as a simple way to do basic queries on top of Parquet data. I suspect it's simply a managed service on top of Apache Drill (https://drill.apache.org/).

Agreed - that would be much appreciated. Very similar to the "Region" dropdown where you get limited to everything in a specific geography.

I've started tagging resources and doing a filter/sort but it would be much nicer if I could just do it once during a session rather than filtering on each service's screen.

Yea - that was my first experience using it for some of the news sites and I ended up spending a weekend redoing my entire site to do pure AMP for desktop as well. The general idea of AMP is to avoid forcing a "rerender" of the DOM so it tries to do as much processing beforehand in order to only render the page once. This is why you have to specify the CSS inline and associate dimensions to each image.

Glad you liked it!

Ah interesting. Thanks for bringing this up. I use uBlock but didn't have any issues rendering it. Since I don't really care about the SEO/Google hosting it it might make sense to just link to my own locally hosted version to avoid the adblocking block.

The direct GitHub link: https://github.com/dangoldin/aws-billing-details-analysis

Earlier this year I spent some time moving my blog to AMP and so far I'm pretty happy with the move. It's hosted on GitHub pages and I made the decision to go full AMP (including Desktop so I never went the /amp/ subfolder route) and allowed me to convert every existing page to AMP without having to have a new set of URLs.

It's noticeably faster but there are a few hiccups I had to deal with - JS needs to be in isolated iframes and getting Disqus comments took a bit of time but in my case the performance benefit outweighs it. At the same time I could just simplified my design without going full AMP and it probably would have been just as quick. The biggest frustration is that if you make any CSS changes it has to rebuild every single page since the style needs to be declared inline rather than as a link to a CSS file.

For those interested: http://dangoldin.com/

On a somewhat related note I started just shutting Slack off at work when I need to get some thing done and it's been incredibly liberating.

Ah got it - thanks for letting me know. Do you know how popular logic languages are these days? I've watched a few talks on Datalog so it seems there's something there.

Agree with you on the SQL part and was just making the point that a lot of the intro Prolog examples can be done in SQL.

Yea - Prolog is super neat and gets you thinking a very different way. It was my first real introduction so took a bit of time getting used to thinking the "Prolog" way.

My understanding is that there's a very limited amount of research going into Prolog so it ends up being incredibly slow for larger tasks. And a lot of it's functionality can be done via standard SQL so there's not a ton of use cases for it.

Pretty cool.

I also did a stint messing around with Prolog and it was pretty awesome discovering an entirely new programming paradigm. I ended up doing the Priceonomics puzzle (https://priceonomics.com/jobs/puzzle/) in Prolog and wrote up about it here - http://dangoldin.com/2013/06/07/fun-with-prolog-priceonomics...

What a shame - it caught a few bugs in my code and has definitely made me a better programmer.

One of the reasons is that the code is hard to maintain with most open source contributions being small improvements. How have other open source projects handled large scale refactors? Is it as simple as someone creating something new from scratch and then duplicating functionality? Are there examples of large open source projects that have had ground up rewrites?

For me that memory was Diablo 1 and finally getting to "The Butcher" - the way they got the environment to match the music was incredible.

Have you tried Brave? I've been using it on the side and it's speedy and constantly improving.

Yea - the comment above cleared it up. Apparently the Cuckoo filter should have rejected the latter inserts.

Ah got it - thanks for the explanation. That makes a lot of sense now.

Neat visualization. One of the nicest things about a Bloom Filter is that you'll never get a false negative (saying something is not inside even if it is). I'm not super familiar with Cuckoo filters but since it allows for removal it's possible to get false negatives.

You can replicate that by entering "cat" 5 times and then removing it 4 times. Cuckoo will indicate it's not in the filter despite it still showing up there a single time. I guess that's the "limited counting" aspect of it. Really cool.

Another way is add /amp/ to the URL before /articles (ie http://www.wsj.com/amp/articles/birchbox-cuts-about-12-of-st...).

Sometimes it will do a subscription request but there are ways around that as well if you look at the network inspector to see what it's requesting..

To play devil's advocate Prolog is considered much more similar to SQL than any other language and I suspect that will have an extremely high learning cost. That may be me being biased due to learning procedural languages first. At the same time I consider myself well versed in SQL.

I've been in those shoes and still feel a bit of that. Would love to chat and share some personal notes if you want to reach out - dangoldin@gmail.com

Nice read. Google's been working on AMP (https://www.ampproject.org/) to deal with this. They're applying a pretty strict policy to speed up rendering on mobile pages by trying to be as specific as they can upfront to determine the layout up front.

We've been messing around and looks very promising. It's much faster than Storm so at some point we'd love to move to it. Anyone knows how it compares against Flink?

TripleLift (http://www.triplelift.com/) | NYC

We're hiring across the board for both backend engineers (Java, Scala) and full stack (Angular, PHP). Our mission is to help make ads better by allowing websites to remove the distracting and annoying ads and replace them with ones that are more native - similar to how Facebook and Twitter have ads that fit in with their user experience. ' A lot of people bash adtech but if you're interested in diving deep into high performance tech and get experience across a wide range of technologies there are few other industries that can compare. Would love to chat with anyone - even if you're not that interested in the roles - dgoldin@triplelift.com

Your recall is correct - it was an Israeli daycare: http://priceonomics.com/effectiveness-of-fines-for-late-pick...

Ah interesting. I was thinking even if the algorithm didn't change the volume could have had a side effect on it. Do you expose the history of the algorithm?

Something interesting is that the top score seems to have peaked in 2011 with 4339. No score since then broke 4000 and there were two that didn't break 3000.

One interpretetation is that there's a lot more posting going on and it's hard for a single post to maintain a top spot as well as potential changes to the ranking alogrithm. Another option may be less engaged users. Maybe a mix of both.

They might also look at the timestamp between the clicks. If they're all together they might realize it's just someone opening multiple tabs at once. If they're a few minutes apart that may mean something else.

What stuck out is how much emphasis there is on the difficulty of building out support for local policies at a product/service level and pushing the onus onto the host.

On the flip side you have Uber which is succeeding by working through the local policies. The apps themselves have functionality that's location specific (think Eats in NYC but not in NJ).

I do think Airbnb is using the local policies as an excuse and can do more to make themselves more acceptable to cities.

Curious what the operations you're talking about are. Get?