schmatz

There Is No Antimemetics Division is such a creative, page-turning, terrifying read and I'm glad it's getting the author some recognition from the big publishers. It's one of those books that keeps returning to my thoughts every so often and I can't wait to see the improved version next year!

I like the term “Incident report”.

I would highly recommend avoiding Dante Labs. I had a terrible experience with them. There were extreme delays, lasting many months. When I requested the full genome be sent to me, they insisted the only way to do it was a hard drive. This hard drive was sent from Italy to California. On its way, it was seized by customs in Mumbai, who opened it and re-sealed it. The package arrived in tatters, and based on my analysis of the genome with the appropriate reference, it appeared to be someone else's genome. Many of the SNPs disagreed with a later 23andme report I got, which I assume is more accurate.

I was stupid enough to sign up for their SmartGut program. They never clearly disclose how much the test costs if you opt to use your insurance. They’re $2700. Not only that, they make it seem like you purchased 6 tests, but the $2700 charge is per test. Their billing practices are questionable, at least from the perspective of the consumer. On the BBB page for uBiome tons of people are complaining. While I can afford the surprise charge, I feel bad for the many that surely cannot.

Cool stuff. I wonder if we'll see Blockchain technology spread to other commodities. The value prop mentioned is transparency - I wonder how much this will improve with this solution, how much of an issue it was before, and whether increased transparency will cause any issues itself.

BTW, I wonder if this is the oldest organization investing in blockchain technology.

It's a great design! I use a similar setup for my blog and wrote about it here: http://blog.michaelschmatz.com/2016/03/31/how-i-wrote-this-b...

I had a chance to meet Tom Wheeler to discuss net neutrality in the summer of 2014. It was coming off of a bad press cycle for him (John Oliver calling him a dingo), so I didn't know what to expect.

He was extremely reasonable and very receptive to the needs of the tech community and small businesses. I came away from the meeting pleasantly surprised and have been happy with his actions during his tenure. Sad to see him go.

I think this would probably be the best choice after templating the filter

Great suggestion, thanks! I've updated the article accordingly :)

RAM is 1600MHz DDR3. I do have an SSD, but as the memory usage of the program was ~32MB, so I doubt it would have an effect haha

Intel Core i7-4980HQ

I updated the blog post with your suggestion; CDN should be updated soon :)

Great suggestion; I wasn't sure the idiomatic way to template this, thanks for letting me know!

I think what you said about mathematics applies for the rest of science in general - it's more and more of a group effort. That being said, I think there is ample room for innovation in emerging areas like biohacking.

There is also something to be said for analyzing all of the new datasets coming online and trying to gather new insights (this is also the least capital-intensive option!)

The main reason that I like this setup over other solutions is that it's very similar to the way I write code; having one workflow makes things a bit easier!

Is there a particular workflow that you use?

This is a good idea for a blog post! The way we have set up the system ensures that we requeue upon failure and concurrent updates are not an issue.

We are eagerly waiting for Lambda VPC support!

I'm happy you're interested in this sort of stuff; shoot me an email, let's chat :) michael@segment.com

We have a worker consuming S3 events and updating the index. On a related note, we have experimented with Lambda to do something similar; AWS has done a fantastic job integrating their products :)

Sure! We batch together data to load into warehouses by time and a few other properties. Usually, to figure out what objects are in S3, you have to issue an S3 list objects command. That operation tends to be relatively slow, especially if there are many objects.

Instead, when we put a new object, we update a table in Aurora which tracks all of the relevant objects. That way, we can query information like "what objects were uploaded in a certain time range" very quickly.

I also linked to the code of the winners in the blog post (you can also find them here https://gist.github.com/schmatz)

Also, could you email me at michael@codecombat.com with details on what stats you'd like to have access to?

That's a great idea! I'll add it to my todo list.

Right now, there isn't a nice and well documented way to do this as we haven't ever had demand for an API like that. You can reverse engineer the POST for the code ranking to submit your code and the GET for the leaderboard information.

Good catch! I'll have to send some feedback to the guy who runs the blogging platform.

Hi malloreon,

You can find the link to the main site in the middle of the top bar of the blog where it says "Click Here for the Main CodeCombat Site".

Yep, they could.

If people start doing that to mess up results, we have ways of remedying that situation. We haven't had to deal with any cheaters thus far as our users are pretty awesome (and there really is no reason to cheat).

We'd probably send each game out multiple times and require consensus or run all the simulations ourselves. I don't anticipate having to do anything of that nature, but it's interesting problem to think about!

Hi michaelmior,

Those simulations are used to calculate the public leaderboards during the competition. They aren't used for final rankings as the simulation results need to be validated. However, they provide a nice way for people to gauge their standing.

To perform the final ranking/validation, we'll play n of the top games on each team against each other and then sort by wins and losses. We're not quite sure of how large n will be, but I think it will be 150-250. It would be nice to calculate the public leaderboards this way, but as a O(n*m) calculation, it really doesn't scale in practice (we have to strike a balance between accuracy and responsiveness of the ranking system). The public leaderboards are calculated with a Bayesian algorithm.

If you have any other questions, feel free to ask! :)

This is so cool :)

That's quite impressive! It is apparent our analysis of GitHub events didn't account for "super-repositories" like this, as this would be the fastest growing repository on GitHub by a large margin, dwarfing large projects like Linux or Gaia (unless Apache or Mozilla also had these super-repositories.) I'm not sure it's quite an apples to apples comparison though xD

On a somewhat related note, another interesting thing to note is that we found apparent discrepancies with GitHub's statistics for contributor counts.

For instance, take a look at this PR: https://github.com/codecombat/codecombat/pull/372

It was clearly merged. Now take a look at the two possible usernames GitHub could have used to tally his contribution: QelioX (his GitHub username) and Esh2349, the username he used on the commit. https://github.com/codecombat/codecombat/commits?author=Esh2... https://github.com/codecombat/codecombat/commits?author=Qeli...

I looked, and I couldn't find anything out of place with his profile. Our method for growth through unique contributors actually handles this case. We didn't have time to explore other properties of the GitHub data set which might throw our analysis off, but hopefully someone in the future will produce a nifty analysis which does account for them.

Looking at OpenStack, it does look like an aggregate of all of the organization's repositories would be definitely the fastest growing. However, we looked at individual repositories, so that's why it's not on the graph. It would be interesting see the growth of the big open source organizations (GNU vs Apache vs OpenStack etc.)

Actually, the Linux kernel, Android, and Apache are all on GitHub, and were analyzed (though the latter two are mirrors, so I guess not purely on GitHub.)

For projects like Linux, while it may have 3500+ contributors, it got that way over 23 years.

We also made the assumption that due to the social nature of GitHub, its meteoric growth, and the visibility that it brings open source projects, the fastest growing open source projects are on GitHub. We presume this assumption is correct, but don't have the data (or even know where we could get the data) to verify it.

You're right, that's true.

It's not the ideal metric, but it's one that is realistic to compute with limited time and resources. To analyze that statistic using a growth metric like "largest 30 day average contributor gain" across such a gigantic dataset, consisting of hundreds of millions of rows and tens of millions of unique repositories, requires resources and engineer time which few people possess. We certainly don't want to divert too much of either of those things away from development of CodeCombat just for a fun data science experiment!