On Cumulative Advantage and How to Think About Luck
ofdollarsanddata.com
ofdollarsanddata.com
It is a snowballing effect, but most people never cross the initial tough barriers(after 10k, it becomes relatively easy to keep growing). In order to cross it, it either involves luck, or you have to know someone more popular than you who can boost your channel.
I'm thinking you could more or less hire influencers to recommend your channel/content?
But the way YouTube works is a bit more difficult. Higher subscriber counts get you higher CPM rates. Higher CPM rate videos place higher in search. I've actually seen this myself in my own competing videos. YouTube really pushes video CPM rate in search.
So on YouTube, the barrier to entry is extremely difficult. To hit 10k subs I had to do tons of marketing, guest posts, bought some advertising. 10-20k? Just post a video every week. 20-30k? Just post a video every month or two.
2016 was my best year and I made about $500-$750/mo. This year about $100/mo. In 2016 I got ad rates from 20-30/1k now it's 5-8/1k. Channel is a coding channel: http://youtube.com/devfactor
I don't think it's just the cumulative advantage of having more people retweet you (and thus the greater relative ease of getting in front of people) I think it's also learning how to present a constant "face" on Twitter. By the time you're at 1000+ followers you've figured out how to be interesting, how to be consistent, you've streamlined your personal site to point people towards your Twitter and all of this feeds back in on itself.
I've heard that after the magic number of 10k subscribers, you have an open invite to the YouTube space in LA/NYC and can use their substantial production equipment and be mentored.
I imagine the long tail for YouTube starts around 1000 subscribers.
That is highly misleading. She was outed by a person with inside knowledge (anonymous tip), which was confirmed by automated text analysis.
https://www.smithsonianmag.com/science-nature/how-did-comput...
Now, rich-get-richer effects can be desirable in a board game in which the starting conditions are perfectly symmetric and luck plays no role, because the more skilled player is going to the one to acquire that snowballing advantage. But in asymmetric games, or games in which luck creates asymmetry, rich-get-richer effects obscure the relative skill of the players.
(1) unless it’s an educational game aimed at demonstrating the effect.
It was intended to have a rich-get-richer side effect. The original intent was to demonstrate this phenomenon.[1]
It's almost impossible to design a game with an economy where this does not happen, unless you mimic real-World societies where they have avoided it (e.g. Scandinavian countries).
This is not a popular point to make to most Americans, but that does not make it any less true.
[1] http://www.bbc.com/capital/story/20170728-monopoly-was-inven...
Which is itself, perhaps, an example of this effect at work. Monopoly is boring to play and contains very little engaging gameplay [1]; a winner quickly emerges and the game then becomes a long grind in which everyone just wants it to be over. There are so many board games that are simply better and more fun in every way, yet time after time someone pulls out a Monopoly set for no good reason other than it has some kind of unfair, unjustified mindshare advantage.
[1] If one plays by the actual rules, it's thankfully over a lot faster, but everyone seems to play by rules that force it into a long, painful endgame.
While that sounds good in principle, it doesn't always work so well in practice. I have played board games that were designed that way and I began to realize that my choices early on did not matter very much. This had the effect of making the whole game feel pointless until the last turn where somebody would quickly grab the win.
These two dynamics (endgame vs snowballing) are not necessarily coupled that tightly (unless in the hands of skilled player which is the whole premise).
It is the problem of it "as a game", but it was the original goal design. It is a lesson on how capital can accumulate and leave people without options.
Monopoly is derived from The Landlord's Game, which was created by Elizabeth Magie in the United States in 1903 as a way to demonstrate that an economy which rewards wealth creation is better than one in which monopolists work under few constraints[1] and to promote the economic theories of Henry George and in particular his ideas about taxation. (From Wikipedia)
Does merit have similar increasing characteristics?
Does it apply in non-competitive contexts?
Market success, notoriously, has non-merit factors: a comment on reddit has to have merit to get a few votes. But to go from a few to thousands, it has to be early, visible in the thread, the thread has to be visible, the story has to be visible on the front page, it has to stay there a long time. ("karma train")
One of the sub-factors is people like to read what others read - regardless of quality - in order to be on the same page, get the same jokes and references etc. To know what's going on - like the news.
In today's era where reviews and award often consider many factors besides just the book quality alone, they've become quite useless. So how are people supposed to find new books? You end up going from a handful of reviewers who tend to like the material you like, or by going to the same author over and over. In either case, it takes new entrants some significant amount of time to reach their full potential. I don't think this inherently means the market is broken.
You probably mean reputation has some positive correlation with merit, so that reputation can be used as a predictor of merit. And of course, you're right.
However, I wasn't addressing how consumers can ascertain merit, but whether merit has similar increasing chacteristics to market success, as described in the article.
That is, if someone starts with a little extra merit than someone else, does it also compound as they practice?
I wanted to exclude competitive factors. For example, Gladwell's Outliers claims that Canadian hockey players have birthdays biased to part of the year, and that this was because the slight age difference makes child players more successful against their sub-year younger opponents in the same age-group. They get more practice and confidence, and get selected for extra training. But, this competitive aspect gives an artificial advantage to players who are slightly relatively better. And also, this advantage is not present just once, but every time they play, for years.
BTW both the points you and I made regarding the merit-market relationship are mentioned in the literature posted by gwern https://news.ycombinator.com/item?id=17052074 (last link):
> in part because people use the popularity of products as a signal of quality—a phenomenon that is sometimes referred to as “social” or “observational learning” (Hedström 1998)—and in part because people may benefit from coordinating their choices; that is, listening to, reading, and watching the same things as others (Adler 1985). - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3785310/
Edit: I'm sure it's a big factor in app stores.
It actually puts me off writing some OSS libraries as it’s hard to know if anyone will appreciate them. If you already have an audience you know you’ll get some feedback and initial users / contributers.
But I think it was more important that Torvalds was the leader of an important software project, could mandate its use, and recruit maintainers, despite the project’s immaturity in the early stages.
Torvalds’ skill did matter, though. He understood the essence of git’s predecessors, and wrote a credible system that had very good performance from day one.
There are a lot of open source operating systems that are foundationally much better suited to the desktop than Linux, for instance, but no one uses them because they don't have Linux's driver or application library, which is because they don't have enough contributors to catch up, which is because no one uses them...
If you have these two things, you get 80-20 like distributions, you get the explanation for why winners keep winning. If you are interested, you can find my simulation and analysis at
http://www.cs.toronto.edu/~arnold/research/80-20/
Kind of shocking how well this works. The intuition is, why has coke won, well they had some initial advantage, and so they won a bit. Now that they have won a bit, they can finance themselves into more competition. For example, they can place themselves into more stores, into more restaurants etc. Now they get a chance to compete more.
Do you see a good way to factor in these things?
r1) Actors have normally distributed abilities,
r2) Actors are chosen randomly based on current winnings, the more you have won, the more you compete,
r3) Winner of competition wins one point from the loser,
You get interesting results, for example, in the two columns below, the left is Household income in 1970 broken into quintiles. The right column is simulation results.
4.1% 6.7%
10.8% 11.5%
17.4% 16.0%
24.5% 23.3%
43.3% 45.6%
If you run the simulation with different rules, the real world quintiles do not match the simulation quintiles nearly as well. You can tweak the simulation to see this as well.
The simulation can be tweaked to handle cases such as inheritance, so an actor with different ability inherits the wealth of a past actor. It would be interesting to see how many generations it takes to loose the wealth to test the adage that wealthy families loose their wealth in 3 generations.
https://www.theglobeandmail.com/globe-investor/globe-wealth/...
Ill try it and get back to you.
I allowed it to evolve for a single generation, enough time for the top 20% of the population to have 80% of the wealth. I now choose a random sample from the top 20% of the population to follow, lets call them T20.
I now repeatedly
1) pass the wealth of all actors to actors with new, random abilities
2) let the new actors compete for a generation (the same number of competitions we used above)
Result: After 3 generations of steps 1 and 2 above, 80% of T20 has lost almost all their wealth, 10% has lost 75% of their wealth, 10% has done really well, growing it by a factor of 8, due to capable ancestors for three generations.
Amazing, it matches the statistics in https://www.theglobeandmail.com/globe-investor/globe-wealth/...
PS: The add-on[1] I use to avoid redirects doesn't like these medium hosted blogs. These links redirect to medium[2] which redirects to original link and I am stuck in an infinite loop.
[1]:https://addons.mozilla.org/en-US/firefox/addon/skip-redirect...
[2]:https://medium.com/m/global-identity?redirectUrl=https://ofd...
That being said I'm not sure I accept that premise universally. It seems like this is true in domains that are popularity contests: pop culture, politics, etc., but I have no proof of that and have never seen anyone do any kind of comprehensive analysis of it. Maybe I just don't want to believe that success beyond a certain point is random.
Not sure that is correct, because (if you agree that King is good), you do need to be good to be successful, and I think that is what meritocracy wants to accomplish for society: that those who are successful are good, rather than bad. For society, it is not necessary that everyone that's good is also successful, just that the bad doesn't rise to the top (too much).
What it debunks is the (incorrect) corollary that if you are not successful, you must be bad/lazy (or that if you are good you are guaranteed to have success). Which of course does not follow (A→B does not imply not(A)→not(B)), particularly if there are multiple necessary factors for success, such as for example quality AND luck/path dependencies. If it is quality AND luck, then it's not the same as lotto, though the random part does play a role.
And yes, wealth redistribution makes a lot of sense for societies, and yes, the "no success → bad/lazy" fallacy is often used to justify not having such mechanisms.
For example, a craps table has a negative expected value for the players, and positive expected value for the house. But on a winning streak (these streaks are fairly common, just like losing streaks) the table might lose money if you look at the period of a few hours.
https://www.distinguished.com/Site/programs/hospitality-and-...
https://en.m.wikipedia.org/wiki/Kelly_criterion
This is why poker players who are good enough to play at a certain table level can't maintain it if their bankroll falls too low. At each stakes level the game not only gets harder, but the minimum bet can eat up your bankroll if you get large string of unplayable hands (let alone bad beats or bad plays). What ends up happening a lot is that players will win a significant amount of money at say the $5 table, then try to play the $10 table, and lose enough money they find themselves back at the $5 table. The really unlucky ones may end up back at the $2 table because they may not have moved back to $5 early enough to be able to bankroll that level properly.
I find the economics of poker to be completely fascinating, and when I found out about kelly betting from a HFT friend if mine, it really changed how I looked at the topic.
You also just made a claim without evidence. It would be more compelling with some.
For example, I have several colleagues who started their careers around the same time as me. Some are much wealthier than me partly because they spent considerable time and effort accumulating real estate portfolios (among other investments). Others have accumulated less wealth than me because they spent much of their incomes on ephemeral pleasures.
Also many things cost less for wealthy people. They literally spend less as a fraction of their wealth than anyone else for credit, for example. Wealth will naturally gravitate towards them even if they do nothing more than act like anyone else because of that kind of thing.
Your second point is accurate, but not really a rebuttal.
A few very unusual people in each generation start with nothing and become billionaires, but for most of the population a running start will get them much further towards a comfortable life than almost any combination of luck and talent.
This is a problem, because many of our social myths suggests that the most reliable and predictable way to become richer is through hard work - and therefore rich people are more socially valuable than the poor, who stay poor because of laziness.
Both of those beliefs are absolutely untrue.
What the OP was pointing out was that, there are people who understand money and people who don't, and that plays just as big a role as having wealth (see lottery ticket example cited in sister comment). I think I've demonstrated I understand your point; do you think you really understand that one?
Also, > the most reliable and predictable way to become richer is through hard work
Do you know a more reliable and predictable way to become richer? It may not be fast and it may not be fair, but in a capitalist society, I believe it to be true.
People who were never exposed to having money to begin with don't know what to do with it. Having money from the start (rich parents) gives you a head start.
https://arxiv.org/abs/1802.07068
" The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking. Sometimes, we are willing to admit that a certain degree of luck could also play a role in achieving significant material success. But, as a matter of fact, it is rather common to underestimate the importance of external forces in individual successful stories. It is very well known that intelligence or talent exhibit a Gaussian distribution among the population, whereas the distribution of wealth - considered a proxy of success - follows typically a power law (Pareto law). Such a discrepancy between a Normal distribution of inputs, with a typical scale, and the scale invariant distribution of outputs, suggests that some hidden ingredient is at work behind the scenes. In this paper, with the help of a very simple agent-based model, we suggest that such an ingredient is just randomness. In particular, we show that, if it is true that some degree of talent is necessary to be successful in life, almost never the most talented people reach the highest peaks of success, being overtaken by mediocre but sensibly luckier individuals. As to our knowledge, this counterintuitive result - although implicitly suggested between the lines in a vast literature - is quantified here for the first time. It sheds new light on the effectiveness of assessing merit on the basis of the reached level of success and underlines the risks of distributing excessive honors or resources to people who, at the end of the day, could have been simply luckier than others. With the help of this model, several policy hypotheses are also addressed and compared to show the most efficient strategies for public funding of research in order to improve meritocracy, diversity and innovation."
While it is useful to understand the nature of how markets operate and the role of luck in them, I find that worrying too much about this is harmful for my personal development and that taking a stance that is something like "luck doesn't exist" is more useful. http://ssygen.com/posts/luck
It's not a binary choice between 0% luck / 100% power, and 100% luck / 0% power. There's a whole continuum in between, and the exact luck/power breakdown for any given action is a matter of objective fact. Where we run into trouble is when our perception of the breakdown does not match the objective reality, and we waste our time on impossible tasks or don't bother to try tasks that could have benefited us.
And the breakdown of luck/power is objectively skewed in favor of power for those who are white, and skewed in favor of luck for those who are minorities. So while white privilege and white supremacy might both be wrong, they are certainly not wrong to the same degree. While privilege is a view closer to the objective reality and is thus probably overall beneficial as long as you don't take it too far. Likewise for feminism/incel.
You really think they’re not trying hard enough? What is trying hard to you?
I don’t know if you’re American but it’s not unusual for black Americans to work twice as hard as their white counterparts to receive equal recognition, or success.
I'm from Brazil if that matters.
Black Americans are mostly native born. Asian Americans are mostly immigrants or just one or two generations removed from immigrants.
Immigrants from Asia can pretty much only get a visa through some kind of work visa. That means they are pre-selected for having a highly-demanded skill set before they can even enter the country. It also takes decades for them to qualify for permanent residence, so they need to be employed in that field for the entire time. It is almost guaranteed that these people will be financially successful because someone who is employed for decades in a field that can qualify for one of these visas will be financially successful. Anyone who isn't will not legally be allowed in the country.
US law pretty much guarantees that Asians on average will be more financially successful than average because they are much more likely to be here on a visa that requires financial success.
I think you can't discount the fact that Asians simply have a better mindset that focuses on hard work more than other groups, which explains at least part of their success. And it's really not a surprise to think that if you value hard work and you act on that value by working hard, you will be rewarded. There's no need to make up excuses by blaming systemic racism or any other sort of external influence that doesn't play as much of a role in your life as your own effort will.
Isn't part of the whole immigration debate in America the fact that Latinos find it easy to illegally immigrate here? Asians literally have whole oceans to cross to get here. Many Latinos can get on a truck and just drive across the border.
You also can see huge income differences between African immigrants and native-born African-Americans. This has less to do with mindset and more to do with the selection bias TheCoelacanth mentioned. Immigrants from Asia, India, Africa, etc. will be more financial successful because they are deliberately selected for.
Black immigrants in the US have roughly the same level of financial success as Asian immigrants, they are just a much smaller portion of black Americans than Asian immigrants are of Asian Americans. The real difference is between immigrants and non-immigrants, not between black and Asian people.
Also, it's untrue that immigration law applies equally to all races. There are many aspects of immigration law that are different based on national origin. People from India and China are excluded from many ways of immigrating except for work visas.
Do you agree that it's best to promote a message of "don't blame others for your failures" over a message of "your life is mostly dictated by luck"? I don't see how you could argue that the latter is better than the former.
And I really won't change my mind on this notion, given that my parents started up poor in Brazil and worked their way up to provide me with a reasonably good life, way better than the ones they had. And the main thing that enabled them to do that was their own mindset about life and just a lot of hard work.
The same applies to many other people I know, and the opposite also applies to many other people I know. There are people who simply don't work as hard, or don't pay enough attention to spot opportunities, or aren't financially responsible, or are alcoholics, or any other number of problems that are self-made. Those people keep themselves trapped in a cycle of poverty through their own actions, not anyone else's.
That may be true, but what’s the effect on your understanding of other people, society, and how the world works? Is that not important to your personal development?
> And so the more general formulation of "luck isn't real" would be something like "you shouldn't blame external factors for your failures".
Well, sure, but luck is still real. How can knowing a true fact not be useful?
> Ideas that empower you should be accepted, ideas that remove power away from you should be rejected.
How about we don’t accept/reject ideas based on how they make us feel but rather based on if they’re true or not. I understand that it is effective, but it will ultimately cause you to reject something vital.
I understand how it works. 1) What you think affects how you feel. 2) How you feel affects your life, how hard you try and how you deal with setbacks.
The article that you linked to about empowerment talks about empowering others by focusing on the equally objectively true narratives where the others have a say in the outcome. But I think that only applies to when you’re writing a comment or a blog post and you don’t have infinite time to talk about everything and if you have to focus, might as well be on something constructive.
If we’re talking about living with yourself, then I think there’s plenty of time to consider all of the perspectives. The nihilist can come to terms that there is no meaning in life provided for him and create his own.
The main problem with adopting certain mindsets is that they have the ability to change the way you filter the world without you realizing it. So even though you can find a way to live with that mindset, if it's a mindset that focuses on things you have no control over, it will be less useful than another mindset that focuses solely on things you do have control over. That's the main point I was trying to get across.
On a tangential note, it would be amazing if the guild/union of authors collectively agreed that all books will be published anonymously for the first year, with the author's name only revealed after the first year anniversary. This would allow newer authors to step out from the shadow of the established ones. I doubt this will ever happen of course, but I think we would see so much more dynamism in the industry if it ever did.
Yes, the readers aren't "losing" anything. But nevertheless, they have still missed out on something they would have enjoyed.
I read the "Robert Galbraith" books. I would have never read it if I hadn't known that J. K. Rowling wrote the book. But I wouldn't have felt like I was missing out, I would have just read some other detective novel instead and been equally as satisfied.
The much, much more likely outcome would be a massive surge in interest for textual similarity analyses.
Yes the market recognises this behaviour and tries to push past success because they get a great return, but familiarity and noise floor are powerful.
For instance, we know small companies often win against much larger ones.
We do? They do?
It seems to me that this statement requires context in order to say much of anything. I don't know what to do with it as a standalone general statement.
Do they win when the far larger business actually cares, in head-to-head competition? I doubt that happens very often. When counting the successes there also is significant danger of survivorship and selection bias because the David vs. Goliath story resonates so deeply with human listeners.
Or do they win when the larger business doesn't even know it is in the race, like an ant screaming at an elephant oblivious to its existence "this pebble is mine"? I'm sure the chances of success here are pretty good.
How did the giants of today get to be giant in a 20-year span when they started out tiny? And why didn't the giants of 20 years ago use their cumulative advantage to get even larger?
This will piss me off now whenever they offer a table in the corner.
It is the way of heaven to diminish superabundance, and to supplement deficiency. it is not so with the way of man. he takes away from those who have not enough to add to his own superabundance.