Hacker News Reader

mendicantBon Data on the uselessness of LinkedIn endorsements

Hope it was helpful.

It's unclear what you t-tested here. Ideally, you would test for difference between groups of "Is there a difference in number of endorsements between people who got a "yes" in advancing to the next round or not". As a followup, is there a difference between those who's preferred was most endorsed or not?

I'm a bit stunned that you didn't recognize Language as programming language...... :(

As an example, people probably get endorsed for SQL or CSS far more than their programming language of choice that is tested in an interview.

mendicantBon Data on the uselessness of LinkedIn endorsements

First 200 pages

https://www.amazon.com/All-Statistics-Statistical-Inference-...

mendicantBon Data on the uselessness of LinkedIn endorsements

If you breakup the plot by any number of categories of technical ability, then there are trends. But to your question, what I was suggesting was that the aggregations done, especially with categorical data that is averaged, are very susceptible to this. And those clusters are reminiscent of situations like this:

https://www.researchgate.net/figure/256074671_fig3_Visualiza...

I'd more so like to see this analyzed against who got to the next round (their binary signal), or yes against preferred language, which I suspect will be much more telling.

The takeaway from that plot is, there is more to the story.

mendicantBon Data on the uselessness of LinkedIn endorsements

Your analysis is extremely flawed.

I really appreciate your sharing the data with us and I like your service. But, this is a poorly done and a far from subtle plug of your business at the cost of LinkedIn.

1) Technical ability vs # of endorsements

Jesus. Hiding stats that you don't like through aggregations? And please read up on Simspons Paradox, which is clearly the case here just by looking at your plot. Try a basic t-test, or rather some statistical rigor, the next time you try to make conclusions from data.

2) Most endorsed vs Language of Choice

As pointed out, this is not the way to frame your problem. By obfuscating what's happening in your histogram (which isn't technically constructued right either) you are again hiding what you dont like through aggregation. By the way, language matters greatly here, and you'd have benefitted by standardization.

3) Your conclusion

"After running some significance testing, though" and not posting your results or methodology, which is at best questionable after reading your analysis.

Again, I enjoy your service, but blog posts on technical ability that are ironically lacking in technical ability don't really make me want to come back.

PS: A little birdie told me that endorsements are quite strong in predictive power for jobs :)

mendicantBon Microsoft Appoints Kevin Scott as CTO

World class engineers (especially data eng) who built some amazing open source things, but internally was a mess.

The reason LinkedIn is a terrible product is because it was mired internally by chronically poor engineering leadership and product politics. Nothing ever got shipped. This new rollout is nice but 5 years too late.

I'm really hoping the MSFT buy will change things, as I really believe in the product and there are some great engineers there.

mendicantBon Ask HN: Who is hiring? (August 2016)

Looks awesome, are you hiring data scientists?

mendicantBon The Wolf

This is how you train soldiers. Some aren't cut out to follow.

mendicantBon The Ultimate Challenge for Recommendation Engines

Honestly, the paper wasn't bad, but I'm shocked they let this article get out.

Combine the author's incredibly poor understanding,shit link bait headline, and baseless article with ZERO results or novelty and this is 10 minutes of my life I want back.

mendicantBon An App Store for Algorithms

Fair enough. Otherwise, does that help shed any light on this side? Thoughts?

mendicantBon An App Store for Algorithms

No, you just had a bad point that you'll come to realize if you ever have to manage a team or company.

More often than not, you outsource items that are not the highest value projects your resources could be spent on. Or, you simply cannot afford to hire a specialist for a specific need. Try to find me experts for some of the pattern recognition algorithms offered who will work at your shitty startup, your shitty big-co with minimal tech wing, or your random mid-sized in the middle of nowhere. I just described 95% of companies on the planet.

And to second relix's point, try to justify implementing something that may have taken years to optimize, even if you think you can do it better. Also, do you want to maintain it? What happens if/when the company loses you? Would they rather have some poorly documented, poorly optimized in house version with nobody around who knows how it works or have a contract with another party they can count on for maintenance and support?

Is this always the case? No. Sometimes it's a core function you want to develop in house. But, it's usually considered.

As a developer, a big step in growth is realizing how to best spend your time and resources, and realizing that contracting implementation of X to further speed up building Y is a choice you'll often have to make.

mendicantBon Inside Google Brain

Bingo. More data beats better algorithms, see

http://anand.typepad.com/datawocky/2008/03/more-data-usual.h...

mendicantBon Inside Google Brain

That's another good point. The required computing power that is now cheap and widely available has changed our ability to even try these methods.

mendicantBon Inside Google Brain

Every model's output quality is dependent on the quantity of data it ingests.

Statistics developed as a science because of the need to overcome the weakness of large samples being expensive. Machine learning has taken off as a direct result of the field's ability to take advantage of and get serious performance gains from the massive amounts of data being generated and leveraged recently.

Here is the best summation I can reference, and I can tell you from personal experience it is very true:

"The accuracy & nature of answers you get on large data sets can be completely different from what you see on small samples. Big data provides a competitive advantage. For the web data sets you describe, it turns out that having 10x the amount of data allows you to automatically discover patterns that would be impossible with smaller samples (think Signal to Noise). The deeper into demographic slices you want to dive, the more data you will need to get the same accuracy."

http://www.quora.com/Big-Data/Why-the-current-obsession-with...

mendicantBon How a mathematician constructed a decision tree to solve a medical problem

They do [1], but not as often as you expect. Most games don't actually use AI, as the goal is to make in game characters appear intelligent, rather than actually being so.

[1] http://www.sauropodstudio.com/dev-diary-number-fifteen-ai-ba...

mendicantBon The Google career path, Part 3: Performance reviews and promotions

Facebook. See http://www.quora.com/For-top-technology-companies-what-of-th...

mendicantBon Artoo, the client-side scraping companion

Looks really awesome, I'll await it's arrival.

mendicantBon Developer Anxiety, we’re not alone

What I've learned in the months since is that in order to overcome the panic and anxiety you have to do something very simple yet seemingly incredibly difficult; you have to let it happen. Let the panic wash over you

I must not fear.

Fear is the mind-killer.

Fear is the little-death that brings total obliteration.

I will face my fear.

I will permit it to pass over me and through me.

And when it has gone past I will turn the inner eye to see its path.

Where the fear has gone there will be nothing.

Only I will remain

mendicantBon 2009, SF is the worst run big city, has anything changed?

Thanks for the tip, I don't think I can change it now (correct me if I'm wrong), and will keep this in mind for future submissions.

mendicantBon Are You Really Gluten-Intolerant? Maybe Not

In the same aggressive tone you're holding towards the article, you probably should have read the publication before criticizing the experiment.

Quick lesson on pharmaceutical studies: they are extremely expensive and often can only have a limited sample size.

Quick lesson on statistics: it was developed to deal with small sample sizes and allow you to generalize to larger populations.

As such, pharmacy has very strict statistical standards with how to setup, design, and analyze experimental results. I'm a statistician, and I was impressed by how well designed this experiment was.

More so, the study only aimed to find a more precise cause of the issues. And, even with the solid analysis they had, are simply saying, "Hey, we should question this anti-gluten thing, it might not be as big an issue as we think".

Raise more questions, that's good science.

mendicantBon Enough Machine Learning to Make Hacker News Readable Again [video]

I believe you are repeating what I said (?). All of the algorithms have different methods of arriving to an objective function and leveraging it's results. Yet, most share the same problem in terms of optimizing it, and yes, most choose other routes.

mendicantBon Enough Machine Learning to Make Hacker News Readable Again [video]

Copped out when called out. Deleted comment said GAs were bad search algos and tend to get stuck at local minima.

mendicantBon Enough Machine Learning to Make Hacker News Readable Again [video]

This is not a good generalization. I've usually only seen this issue with optimization problems when:

1) You haven't played with parameters 2) Implementation is not correct (usually the case with genetic algos, since it requires a reasonable amount of domain expertise vs say GD)

mendicantBon Enough Machine Learning to Make Hacker News Readable Again [video]

+1 I'd also add that genetic algorithms are for optimization, and can't really be compared with most of the algorithms in that chart. It'd be a sub-level where different optimization techniques for finding model weights, for each type of approach (classification, clustering etc)are compared.

mendicantBon Simpson's Paradox is Back

You're correct. It isn't different. Simpsons paradox is actually a key indicator of a confounding variable.

mendicantBon LinkedIn’s Bad Terms of Service

The post was hardly worth writing in the first place. This is extremely standard social network (or internet company) practice. The privacy concerns of just giving a massive dump of all that data are very complex.

Given that, it's incredibly obvious the post was written as a self plug, and burying it was disingenuous. If that author had begun with, "this is an issue and we have a solution for you", it would have been much more honest.

mendicantBon LinkedIn’s Bad Terms of Service

"Collect them all where it makes the most sense. If that’s iCloud, Google Contacts, Outlook.com or heck, even FullContact, great"

Completely discredit your post why don't yah?

mendicantBon The Absurdity of LinkedIn

My thoughts exactly. He's completely blind.

mendicantBon Weebly Valued at $455M Amid Website-Building Boom

This

mendicantBon Weebly Valued at $455M Amid Website-Building Boom

You're still assuming an even spread amongst the ones that are in that 1M. My point is it's usually hard to tell, and you should assume very few sites hold most visitors. Just keep re-applying the concept (of the 1m, 1000 account for 70% etc).

mendicantBon Weebly Valued at $455M Amid Website-Building Boom

I'm pretty sure this is exactly what I just said.