anton_tarasenko

I tried to do the opposite once: to rank GitHub repos by effectiveness of contributions, where effectiveness is measured as stars divided by the number of contributions. Sort of, best things done quickly. With all cons and pros of this metric, I've got these projects in top 5:

1. ievms: Automated installation of the Microsoft IE App Compat virtual machines, https://github.com/xdissent/ievms

2. resume: Resumes generated using the GitHub informations, https://github.com/resume/resume.github.com

3. impress.js: It's a presentation framework based on the power of CSS3 transforms and transitions in modern browsers, https://github.com/impress/impress.js

4. deepdream: Neural Network art, https://github.com/google/deepdream

5. favicon-cheat-sheet: cheat sheet to favicon sizes/types, https://github.com/audreyr/favicon-cheat-sheet

That would work for most employees. Founders know much more about their company and don't offer equity if things go well.

Equity is a good bargain right before the next funding round — when cash balance is low and founders pay with shares.

In this case, the question is how much the stake is worth now. Ask founders the share price of the last funding round. That's the closest market valuation you can get.

Exit conditions (exercise window, sale restrictions) are a must-know, but secondary. An employee can borrow to exercise options and then sell the shares. His company would love to buy shares/options back because they'll have to consolidate equity upon IPO/sellout anyway.

In general, companies go through so much dilution and uncertainty that worthwhile equity stakes start at 5-10% for early-stage startups.

A polite reminder: some people predict population dynamics for living.

For example, the World Bank: http://data.worldbank.org/data-catalog/population-projection...

If you look for dinosaurs, the WB also have a 1984 report on topic: http://documents.worldbank.org/curated/en/496471468156899142... (see p. 186, "Population data supplement")

The UN, IMF, and most national agencies also release their numbers.

In general, world GDP growth — and its correlates, like HDI — is stable enough to predict population growth. As a country gets richer, fertility falls: women get careers, jobs require more education for children, people start to rely on savings, instead of family. The aggregate numbers are smooth and predictable. At least, for a reasonable time horizon.

But you need to fine-tune the model at the right aggregation level. For example, the US, EU, Japan have similar GDP levels, but fertility in the US remains high. Census data helps settle down these issues.

Lending Club almost went broke in the late 2000s when it experimented with real P2P lending. Defaults on the loans chosen by lenders went up, so Lending Club had to improve its underwriting model and effectively exclude the lender from the decision making process.

Equity crowdfunding is the same as P2P lending but more volatile (since equity is a claim on everything what's left after paying everyone else). It's harder than lending. And we haven't seen successful examples of P2P lending yet.

Another thing is, we had crowdfunding in 1929. It turned out badly, so now we have the SEC and many restrictions on equity offerings.

China's saving rate remains at 45% of GDP. Most other countries have it around 10-15%. Savings have to go somewhere, and this is why China accumulates debt. Every yuan of savings turns into one-plus yuan of debt. The current debt ratio adjusted for savings is comparable to the US's, for example. Unlike US, they put it into infrastructure because people don't spend growing income on consumption.

To put it simply, the US worker gets a salary and buys a car, while the Chinese worker makes a deposit in a bank for a construction firm to borrow and build a railroad. Both cases describe transportation services, but they look very different in national accounts.

As for the exchange rate, it's not that relevant for local investments since China's going to pay in yuans anyway.

Slightly relevant:

- MIT Technology Review Emerging Technologies Top (2002-2016): https://www.technologyreview.com/lists/technologies/2016/

- Gartner Hype Cycle history since (2000-2016): http://imgur.com/gallery/noBKI

- Google N-gram viewer (using technology as a keyword): https://books.google.com/ngrams/graph?content=xerox%2Cfax%2C...

My take: Economics, economic growth and development.

It's funny that automation is under attack. Any conventional measure says that productivity growth declines and the number of jobs grows.

First, the rise of unemployment is not supported by facts.

The widely cited unemployment rate went down to 4.6%.[1] That's the pre-crisis level of the overheated 2008 economy. This reduction is costly, but manageable.

Jobs disappear, but new ones get created. Nonfarm payrolls (aka "created jobs") are well above zero.[2]

So what's the unemployment the media keep talking about?

The unemployment voter. 4.6% is greater than the winner's margin in any of the recent presidential elections. It's a valuable asset to candidates and, thus, a hot media topic.

The average voter actually gets better thanks to productivity gains, not employment (which we can't reduce below 4% anyway). Productivity accounted for 2 percentage points of per-hour output growth.[3] It still contributes at least 0.5 pp. This decline is the real problem: losing those 2% means being poorer by 21.9% by the end of the decade.

But productivity doesn't vote, so this loss is going to happen.

[1] https://fred.stlouisfed.org/graph/?g=4CaW

[2] https://fred.stlouisfed.org/graph/?category_id=0&graph_id=12...

[3] Gordon, “Perspectives on The Rise and Fall of American Growth.” p. 73.

Target research-intensive jobs. Job descriptions are clear about what sort of machine learning professionals the company is looking for. In most cases, they want data engineers and developers who can make big, scalable production. Not researchers.

Just find who's looking for economists, specifically. Consulting firms and government agencies hire economists, but they don't call them data scientists. Investment funds may be interested, though hedge funds prefer math and physics graduates because they are easier to train.

Professors don't care about high prices. Universities pay for subscriptions. The pricing is political in Europe where education is funded by the public. US universities pass high prices over to students. Students pay with debt since good universities have oversupply of applicants anyway.

Elsevier had been buying academic journals for decades. A typical scheme is like Cell's story.

A professor establishes a journal under a big university's publishing arm. Then the professor thinks how to make money. Elsevier makes an offer and the professor accepts it. The journal becomes the property of Elsevier and the editors keep reviewing papers for free because it's good for their CVs.

Looking at older HN posts[1], Elsevier becomes another Comcast. That said, boycotts have not reversed the Group's profit trend.

[1] https://hn.algolia.com/?query=elsevier&sort=byPopularity&pre...

Yahoo controls 8% of search traffic and costs $4bn. Google controls 64% of search traffic and costs $558bn. That's a 17-fold difference in the price for 1% of the traffic. Even more if you take US search traffic only (to exclude Yahoo Japan).

Verizon can get rid of the costly parts of Yahoo (mostly its media business) and have its own small Google to make cash. That was $1.8bn per year in 2015, while Yahoo's overall revenue stands at $3.9bn (Asia excluded). So Verizon buys the company at 1x revenue.

Who would quit such a deal?

Yahoo's press release with details:

http://www.businesswire.com/news/home/20161214006239/en/Impo...

$10 is the "willingness to pay" (the number is in the article). There's no good estimate of drone delivery costs since the interested parties cherrypick locations and distance.

That's a last mile solution.

Comparing to human costs of Amazon's preferred shipping method: USPS's subsidized parcel injection. The last mile belongs to USPS. In the urban environment, that's about 15 minute per parcel at $20 per hour (labor plus trucks). I.e., $5 per parcel. For FedEx, the cost of the entire expedition process is roughly $6.6 per parcel ($43bn cost of revenue for 6.5bn deliveries), so the last mile costs less than that.

Flirtey CEO Matt Sweeny suggests $10 per drone delivery. He doesn't specify from where. I assume he also means the last mile.

Are drones more expensive than humans? Air is expensive. Think of helicopters vs cars. With all congestion, helicopters have not become the vehicle of choice. They're noisy and can land only in certain locations. Most urban areas are too dangerous for them.

Air drones inherit these disadvantages.

As for Amazon, it'd have to stop outsourcing delivery to postal services and build a lot of physical infrastructure. Few tech companies are comfortable with this because they become capital intensive businesses, which they don't like.

So, not Amazon and not air drones. Perhaps ground vehicles within city borders and only after cars learn driving without humans. That's 5-10 more years.

To those who want to know more about quant jobs,

Mark Joshi has a cheat sheet intro to the profession: http://www.markjoshi.com/downloads/advice.pdf

Joshi co-authored the standard reference for quant interview preps called Quant Job Interview. Interview questions with solutions. It's available on Amazon.

If you just wanna try brain teasers, google something like "ext:pdf quant interview". There are many of them.

I've made a similar list for economists. It included a list of practical applications of ML. Developers can get a sense of what the discipline can do before jumping in.

APPLIED MACHINE LEARNING CASES

## Business

1. Kaggle, Data Science Use cases. An outline of business applications. Few companies have the data to implement these things. https://www.kaggle.com/wiki/DataScienceUseCases

2. Kaggle, Competitions. (Make sure you chose “All Competitions” and then “Completed”.) Each competition has a leaderboard. When users publish their solutions on GitHub, you can find links to these solutions on the leaderboard. https://www.kaggle.com/competitions

Industrial solutions are more powerful and complex than these examples, but they are not publicly available. Data-driven companies post some details about this work in their blogs.

## Emerging applications

1. Stanford’s CS229 Course, Student projects. See “Recent years’ projects.” Hundreds of short papers. http://cs229.stanford.edu/

2. CMU ML Department, Student projects. More advanced problems, compared to CS229. http://www.ml.cmu.edu/research/data-analysis-projects.html

3. arXiv, Machine Learning. Drafts of important papers appear here first. Then they got published in journals. http://arxiv.org/list/stat.ML/recent

4. CS journals. Applied ML research also appear in engineering journals. https://scholar.google.com/citations?view_op=top_venues&hl=e...

5. CS departments. For example: CMU ML Department, PhD dissertations. http://www.ml.cmu.edu/research/phd-dissertations.html

## Government

1. Bloomberg and Flowers, “NYC Analytics.” NYC Mayor’s Office of Data Analysis describes their data management system and improvements in operations. http://www.nyc.gov/html/analytics/downloads/pdf/annual_repor...

2. UK Government, Tax Agent Segmentation. https://www.gov.uk/government/uploads/system/uploads/attachm...

3. Data.gov, Applications. Some are ML-based. http://www.data.gov/applications

4. StackExchange, Applications. http://opendata.stackexchange.com/questions/3346/examples-of...

## See also

The original article: https://antontarasenko.com/2015/12/28/machine-learning-for-e...

A related list of cases: https://www.quora.com/What-are-some-practical-applications-o...

The acquirer issues new shares and gives them to the acquiree's shareholders in exchange for their stake.

In this case, the acquirer also buys out the acquiree's debt. This part of the deal is funded by a loan from Bank of America and Morgan Stanley.

To summarize biases further, in one word: Incompetence.

Lab results confirming cognitive biases come from testing small groups of students (up to 200). Among other things, it means: (1) respondents with similar background, so we can't generalize (2) respondents don't care about outcomes, (3) tests are synthetic. Plus publication bias and other standard issues.

These results are themselves a sort of confirmation bias.

Mistakes in real life happen when we don't know what we're doing. If a person can learn, he'll discover systematic mistakes. But that comes with domain experience, not cognitive science.

A similar project: https://www.teoria.com/en/tutorials/

Focused more on practitioners. Includes exercises with MIDI/USB piano keyboards in the browser: https://www.teoria.com/en/exercises/

These resources also can be useful:

1. https://www.musictheory.net/lessons (exercises included)

2. http://www.earbeater.com/online-ear-training (ear training exercises)

3. https://www.iwasdoingallright.com/tools/ear_training/online/ (ear training)

4. https://trainer.thetamusic.com/

5. http://music.stackexchange.com/

N. Wikipedia is a good reference. E.g., https://en.wikipedia.org/wiki/Glossary_of_musical_terminolog...

If someone wants to try music, start with playing an instrument and then pick the theory. If in physics theory says how the world works, music theory is mostly about labeling things that sound good vs noise. And it's hard to get the words without playing first.

Walmart and Amazon alone sell $600bn worth of goods. That's 4% of US GDP. These two pushed thousands of independent retailers out of business due to lower prices (and smaller profit margins relative to revenue).

The same happens in transportation, restaurant business, hotel industry. Firms get bigger since 1800. Naturally, the number of new firms decline. In govt stats, it appears as "fewer startups."

Meanwhile, the economy grows thanks to productivity gains in big companies. So what do we want: productivity or new small businesses?

When I worked with the HN post data, I noticed that some years ago HN users had correctly predicted the "Show HN" projects that later got funding. Those projects had more upvotes.

The more recent data has no such connection. It seems that the influx of users reduced the quality of judgement.

So one way to improve HN submissions and comments is to weight points by the user's tenure on HN.

I also suspect that early comments dominate late comments by the time factor alone. The sorting algo gives a brief advantage to new comments, but old comments are more visible. A post on the front page gets 30+ comments in the first hour, and latecomers can only post into the void. To address that, long branches could be collapsed by default, leaving only 2-4 visible messages per branch.

CAPE (aka price-to-ten-year-earnings, PE10[1]) helps devising a rule of thumb for timing.

Stocks don't deviate from their expected earnings for long, so when the ratio jumps over the median PE10, the market becomes expensive.

Holding cash during the period of high PE10 and investing it in stocks under low PE10 is a good idea for a person who's saving for retirement and doesn't want to become a professional investor.

[1] https://en.wikipedia.org/wiki/Cyclically_adjusted_price-to-e...

Mobile-friendly version:

http://nbviewer.jupyter.org/github/antontarasenko/gpq/blob/m...

Two reasons:

1. Agencies often buy from distributors and integrators, not publishers. In this case, original manufacturers may appear in the variable `descriptionofcontractrequirement`.

2. Spelling may differ. Variable `dunsnumber` is be a better indicator for uniqueness.

Yep, and http://government-contractors.insidegov.com/ is pretty easy to use.

No, it's the new conditions. Some time ago they just sent full Excel files to journalists and researchers.

0. I measure Show HN projects, some of which are startups. The dead/alive status as success is a proper measure for survival. For success, see the section on commercial success.

1. OLS works fine in classification problems. And it has advantages. For example, see http://chrisblattman.com/2015/07/22/statistician-neal-beck-j...

And I purposefully included results from three models, not one. They all say the same thing.

2. Multicollinearity refers to perfect multicollinearity. Having correlated independent variables (as in score and comments) are okay. See http://stats.stackexchange.com/questions/86269/what-is-the-e...

3. I mention R² as one measure of predictive power.

4. The regressions without GitHub-hosted projects return same results.

I've mentioned this issue in section "Comments". If briefly, redirect does not affect statistical results.

I had their Excel export file c. 2015, when it was still free.

Try Wayback Machine to explore them:

http://web.archive.org/web/20101030035241/http://news.ycombi...