Comment by ninetyninenine | Hacker News Reader

Comment by ninetyninenine | Hacker News Reader

Parent Full thread

>The programmers who are both smart and clean don't hang out with you.

Not true. I've literally stated in my post I've seen them. Your statement is illogically contradictory to my statement.

>From your perspective, it looks like there is an inverse correlation but that is only because of Berkson's paradox.

Possible. But I'm not sure how you can state this as fact without knowing where I've worked or having data to back it up. Everyone is going off subjective opinions here and no one can be sure who is definitively right or wrong. I feel you're the one who's biased here. Unless you have something more then anecdotal data to back that statement up I feel it's just your own two cents against mine. Which is fine, anecdotally both are valid.

>They don't get hired at the same companies you work for, they don't have the skills needed to work on the same projects you work on, their articles don't get to the front page of Hacker News and so on.

Right. So what companies have I worked for? You can make this statement because you know? Anecdotal evidence or not, you literally have no basis here.

> But I'm not sure how you can state this as fact without knowing where I've worked or having data to back it up.

Because you haven't met them. Where you've worked is irrelevant. Either a) they exist, and you've not met them because you work somewhere that excluded them, or b) all of the places you've worked hired equally from the pool of all programmers, and didn't happen to find any. without knowing where you've worked, I can state for a fact that where you've worked had hiring standards, explicit or implicit, affected by internal and external factors. Thus, B is less likely than A, despite having no data on where you've worked. It's not possible you've worked at every company in every subfield there is to our industry, and thus you've not encountered every type of programmer there is. Programmers for ransomware groups are gonna be a much different group of individuals than from those programming VR goggle firmware, are different from PhD candidates writing code to never get reused, to programmers for HFT desks, for instance.

There are 4 quadrants:

    .     dumb  |  smart
    messy __1___|___2___
    clean   3   |   4

Your theory is that only 2 and 3 exist, and that 1 and 4 are impossible. The world is wide, and I've met programmers in 1 and 4, with as much evidence as you do, anecdotal evidence. If you do have data to back up your claim, I'd love to be proven wrong! Because while my habits put me into group 2, I'm definitely more in group 1.

But as my sample size grows as I work for more companies by probability the data becomes more and more inline with the actual population.

Who's to say my sample isn't a good representation of the actual population?

Edit: in response to your edit of quadrants. I never said they don't exist. I stated that there are correlations. And I explicitly stated that despite the correlations I've seen examples of all types of programmers. Even programmers outside of the correlations.

Of course as stated before, these correlations are anecdotal. Just my experience. I don't think it's necessarily invalid despite that.

Simply on the basis that sampling bias exists, I'll say your sample isn't a good representation of the actual population. Presuming you're human and haven't lived multiple lives simultaneously, it's just not possible you've worked in every subfield of our industry, meeting every possible variation of programmer. You didn't get your current job from first principles. There's bias, just from job-1 on how you got your current job, nevermind your entire whole background as a human being. A coder in India lives in a wholly different culture than someone in Eastern Europe is different from a North Korean is different than a San Francisican programmer, never mind the actual work they're doing; firmware, web crud, kernel, LLM. Different types are going to be more prevalent in different areas, and unless you've got a truly random sampling of workplaces, there's gonna be some bias in action.

>Simply on the basis that sampling bias exists, I'll say your sample isn't a good representation of the actual population.

I would say it's possible. But also every person I've ever met for the most part has 2 eyes. Despite sampling bias my measurement is pretty accurate to the population that most humans have two eyes.

So just because bias exists doesn't render a measurement inaccurate. It could still be accurate.

In fact I doubt even the most rigorous science can be completely unbiased. You can't say just because bias exists the measurement is inaccurate.

It could be accurate. A measurement existing opens the possibility up for the measurement to be accurate and true to the real world scenario.

For something as basic as number of eyes, you make a valid point, but what's more likely? That your career choices have been free from any bias which makes your sample accurate, or that there's many, many biases, from the culture of where you born to the programming language you first learned, to how rich your parent were, that have lead you to have the career that you have, resulting in a skewed sample?

I think what you say is valid. But I also believe that there is a realistic chance that my sample is accurate despite the biases.

It also makes sense logically. A smarter person will be faster at reading sloppy code.

as a sloppy coder (until I clean it up for review) I appreciate the compliment!

> I stated that there are correlations

The funny part, the reason it's called a paradox, is that the correlation can be fake or can even be reversed. We cannot know either way just from our personal experiences, the selection effects are too strong.

And the anecdote you tell, the part about smart programmers being smart enough to get away with being a little messy, and dumb programmers being obsessed with cleanliness to reduce cognitive load, are exactly the sorts of situations that can cause the paradox.

It's like when you are casually dating a bunch of people and find that the good-looking ones tend to have bad personalities because they are attractive enough that they can get away with it. It is a real correlation from your perspective, but on a global scale the correlation is opposite. Good looking people tend to be more charismatic in real life, we just can't tell while we are on the dating market because the ones who are both good looking and charismatic are out of our league, and as for the ugly/mean ones, we are out of their league.

True. But how do you know the paradox applies here? What if my sample is accurate?

What I don't get is how someone can out of nowhere say the paradox applies in my case. I mean this paradox can apply to any measurement even the most rigorous experiment. So why suddenly come to my post and bring up this application?

I didn't intend to make a statistically rigorous statement or anything, I just thought your anecdote was a funny application of Berkson's paradox.

The fun part is that I don't need to know how smart or clean you are, or how cool the companies you work for are. As long as you subjectively perceive people to be cleaner/messier than you, or smarter/dumber than you, Berkson's paradox will still hold no matter what your objective, absolute levels of those traits are.

For example, it's perfectly possible that the programmers you consider to be dumb but clean or smart but messy are still both smarter and cleaner than me. Berkson's paradox applies to me too, after all, and we don't hang out together.

What if the sample of the people I hang out with are a completely accurate representation of the entire domain? Would the paradox still apply?

When it comes to smart/clean, maybe you can hang out with enough people to get an accurate sample.

But then you would find other paradoxes, like maybe you'd find that the smart/clean programmers you meet have a worse personality or are less athletic than the dumb/messy ones.

The only way for it to not apply at all, for every possible trait, is if you hang out with the entire planet at the same time.

Wouldn't personality and athleticism be off topic though?

I'm only referring to two traits here: intelligence and code sloppiness.

Does this paradox still apply in this case? Or is this paradox some kind of broad thing not relegated to a specific correlation?

They're off topic, but the point is you'll find correlation where you want to.

Possible. But also possible that no bias influenced my observation.

And who's to say that the correlation I happen to want isn't real?

I am. I mean, it could be you're right. everyone's entitled to their own opinions though. some people think the moon is made of cheese, for example. but the reality is the inherent bias in your sample set is too large to be ignored.

My claim from a common sense perspective isn't outlandish as saying the moon is made of cheese.

Biased samples don't automatically invalidate every aspect of the sample as a description of the reality. The possibility remains that every most programmers have two eyes just as smarter programmers tend to write sloppier code.

Look the general thing here is that anecdotal information is not automatically invalid. Your type of argument just goes around dismissing all forms of anecdotal data as if it's completely false. It's like you cannot even get up from bed and put your feet on the floor because there's no science saying that the floor exists.

The plural of anecdote is not data! Looking around at the people surrounding you, you could also easily conclude that everyone has the same skin tone. If you've only ever lived in, say, Scotland, you could erroneously conclude that, on top of everyone having two eyes, every one is also white. It's not until you look past the fact that you're in Scotland biases your dataset to being exclusively white people, that you see there are other kinds of people out there.

Living in Scotland and sampling your data there, concludes that all humans are white, along with having two eyes.

>Living in Scotland and sampling your data there, concludes that all humans are white, along with having two eyes.

So? I also conclude that most humans have two legs and 10 fingers and a head, and two ears and skin and hair. We gonna dismiss all of that because the data is just anecdotal?

>The plural of anecdote is not data!

The plural of anecdote is anecdotes (with an s) the adjective form of anecdote, is "anecdotal" as in "anecdotal" data.

Anecdotal evidence isn't worthless, but it's not solid proof either. It's prone to bias and lacks the breadth and rigor of systematic data. Pointing out common human traits doesn't strengthen your argument; these are well-documented facts, not mere anecdotes. The leap from "my experience" to "universal truth" requires data, not just stories. Anecdotes spark curiosity; data provides answers.

Anecdotes are more then just curiosity. They are real possibilities.

When you walk you assume the ground is real based off of anecdotal data. Not science.

Data and science is hard and highly limited. The majority of the world is unverified with science. To limit your conclusions to only rigorous data is to say you cannot walk because you aren't sure the ground exists.

My claim here is that smarter people are more likely to write sloppy code. My meta claim is that That claim is both based off of anecdotal data and also likely to be true.

Yeah, anecdotes are good for making observations to base a hypothesis off of, but that's it. You can't really draw conclusions from them as if they're data. Walking on the ground is based on a repeatable experience and is based on physics. We can't say if the conclusion you're drawing is most people have two eyes, or that most people are white. Claiming smarter people are more likely to write sloppy code based on anecdotes overlooks how diverse the field of programming is, and how complex programming is, and how different people are. It's a mix of education, experience, culture, and individual habits. Using anecdotal evidence for such claims is going to overlook counterexamples and is basically gonna be confirmation bias.

Your meta-claim lacks rigor. Again, the plural of anecdote is not data. The truth of a claim in the realm of human behavior and skills requires empirical evidence, which involves a stupid amount of work to make systematic observations to get shittons of data to override sampling bias, far beyond what any one human could experience. Without data, we've got speculation, not reliable conclusions.

Everyone's entitled to their own opinion. You're entitled to believe "smarter people are more likely to write sloppy code." Just like I'm entitled to my opinion that that's based on evidence subject to confirmation and sampling bias rather than any grounding in fact. :) We're going around in circles at this point, so I'll stop, but it's been fun.

No not, "we". You are going in circles. I already stated that you walk therefore you believe the the the ground exists without science. You already assume a truth with zero rigorous scientific data.

Yet you don't counter this claim. You make the same argument again and again stating that anecdotal data isn't truth. But you place each foot one in front of the other whenever you walk assuming the ground exists from purely anecdotal data. Why?

Thus my argument is valid. IF you don't respond to this then it is YOU who is going in circles. I have definitively responded to your point here with a contradiction. You HAVE not addressed the contradiction. The opening is laid out by me, but instead of moving forward you decide to repeat the same point.

Not trying to be offensive here. But I am clearly laying out exactly what's going on here. It is YOU who is travelling in a flat circle. NOT me.

Reply on news.ycombinator.com