Computer Scientists Are Astir After Baidu Team Is Barred from A.I. Competition
nytimes.com
nytimes.com
This competition (the "Large Scale Visual Recognition Challenge" aka "ImageNet") isn't just some random competition. This is the competition that gave rise to the recent explosion in interest in Neural Networks.
In recent years Google, Microsoft and Baidu have been one-upping each other to the point now where they are getting very close to better-than-human performance (ie, humans disagree with other humans more often than their systems disagree with the average human rating)[0].
Andrew Ng went to Baidu to start their team. I don't believe he is still involved in this challenge.
Baidu has been getting close and close to Google's performance. Every year so far Google has topped it at the right time, but Baidu has later passed the Google benchmark that year.
There were reasons to think that this could be the year they finally beat them. [1] is a story from January about the system that Baidu had built then.
Now this.
[0] http://karpathy.github.io/2014/09/02/what-i-learned-from-com...
[1] https://gigaom.com/2015/01/14/baidu-has-built-a-supercompute...
I wonder why google has that new image categorization captcha when it's a task where computers fare better than humans.
Then they shouldn't have cheated. This is the same company that didn't do anything to stop it's network being used to wreck GitHub for a week (and yes, it's possible to avoid the GFW injecting JS onto your site - buy space in a non-China DC).
This brings quite reasonable questions about their past performance.
I don't think this second claim is necessarily derivable from your first. Yes, they were caught cheating... By making multiple submissions of their own systems in order to faster derive which is the most promising. IMO, the performance they have achieved is still theirs to claim, because in the end they did create a system that reaches that performance.
Ideally, your final test data set should be completely separate from training data. Because even limited exposure quickly renders that test data meaningless.
PS: In the end it's much like taking the same test with word for word identical questions a second time. Yes, it's a test of something, just not the intended subject matter.
Science is based on a huge deal of trust and violating that trust for your own short term gains is inexcusable. Even worse, any honest scientist will have find it more difficult to improve upon results that can be explained by overfitting.
A good short description on several issues that plague the field is "Clever Methods of Overfitting".
So now the record is probably more like 4.5% than the 4% or whatever Baidu's final paper was claiming. To keep this in perspective, that's a few months or a year of progress, the Imagenet record is still human-equivalent, and deep learning is amazing.
I think they've all surpassed the human-level baseline by now. There's some stuff in there, like telling different dog breeds apart, that humans are not very good at.
Baidu's blog post on the result has also been removed. Their ImageNet team was led by Ren Wu. http://webcache.googleusercontent.com/search?q=cache:Fs_26P8...
There was an interesting story recently about how a disproportionate amount of Chinese students are expelled from US universities for cheating.
http://blogs.wsj.com/chinarealtime/2015/05/29/u-s-schools-ex...
The problem is so entrenched that some students rioted when they were prevented from cheating, because they felt that this put them at an unfair disadvantage compared to other schools where cheating was tolerated:
http://www.telegraph.co.uk/news/worldnews/asia/china/1013239...
Arguments that it is stealing are also nothing more than rationalizations. That's what both sides wield: it's a war of rationalizations.
(I tend to be more convinced by the "it's not stealing" rationalizations myself, simply because stealing an object makes it unavailable. If someone stole my bicycle, I'd be upset. If they cloned it in a Star-Trek-style "replicator" and took the copy, I couldn't care less. If you lend a book or DVD or SD card or whatever to someone, upon it being returned, you have no way to test the hypothesis "a copy of this was retained by the borrower". Not testable is not real, for all practical purposes.)
The other side is a person who is capable of rationalizing theft because it is a minor theft.
The theft isn't of an object, it is theft of compensation due to a creator. Duplication isn't the issue.
Spinning it as purely a compensation issue is being a bit reductive, IMO. And from there it leads directly into objectively unanswerable intent questions.
Also, since DNA controls what you look like, including your silhouette, you will have to pay me every time you allow a light source to cast a shadow of yourself onto a flat surface.
Also, when you take a bath, you displace a volume of water corresponding to your shape. That is an unlicensed copy.
Your footprints in the dirt or snow, ditto, not to mention the fingerprints you leave on everything you touch.
---
Man, think about it. Millions of years of evolution (based on continuous unrestrained copying and alteration on a massive scale) lead to Man. What does man do? Put three guitar chords together with some puerile, hackneyed lyrics, and insist on compensation from everyone that hears the crap.
Based on my limited understanding of copyright law, it does in fact concern itself with actual duplication, and is rather mum about compensation.
You can infringe by, say, altering something that is freely distributed in a way that is not permitted by its license (for instance changing the copyright notice to say that you created that work, rather than the author). The author could be adamant that permission to do that is not granted at any price (so that it's not an issue of compensation, but purely about unlicensed duplicates with unauthorized alterations).
Breaking into a house - we have a name for that - it's called "breaking and entering" and "trespass".
However, going along with your analogy, say you're in my house - from my perspective, I would care about what you did.
If you took something that was mine, and denied it to me, I'd be quite annoyed. (I've had a expensive road bike stolen, so yes, I know what it feels like).
However, say you took photographs of everything I owned, to copy my packing style and organisational techniques? Or took them so that you could re-buy everything I have for yourself off Amazon? I probably wouldn't be annoyed, so much as think you odd...
Your example of how it's "practically" impacting your life is that if you happen to commit a felony against the person and then analyze their computer explicitly looking for it, you'd happen to find out?
I think that's the very definition of "not a practical difference in my life", given the distinctly low likelihood that series of events would ever happen.
Would you care to try again?
And now you suddenly want me to pay for it? Why, what changed?
I believe a portion of the cost of blank tapes was allocated to recording artists to cover this costs, so perhaps it wasn't entirely free...
So which one is it: was music free or not?
When an ad comes onto one radio station, you switch to another without ads. Redundant array of independent radio stations. Advertising is damage. So advertisers coalesce their ad timing, so the damage happens on all independent channels simultaneously, so you turn off the radio for several minutes, maybe switching to Pandora or another stream.
When Youtube plays ads, you mute it and goto another tab, and keep watching a different video. When the ad is done (such nice ads for telling you their duration), you simply return to the tab and watch your content.
There's always another service that's looking to hook you before they advertise. So you can just switch between them.
https://www.youtube.com/watch?v=GU7axyrHWDQ
It might be cheating, it might be copyright infringement. Depending on the circumstances it may or may not be moral. But 1000 unauthorised copies do not mean 1000 lost sales. The situation is more nuanced than a simple 1-1 correspondence.
Unlike fraud, the ethics of violating copyright are highly situational and quite complex. A recent example of mine:
I purchased a very expensive publication; my purchase included a PDF copy that was (unbeknownst to me) DRM-locked with very onerous DRM.
- I fired up IDA Pro and broke the DRM so that I could read the PDF without their invasive DRM; I have no intention of distributing the results. This is actually illegal, and in violation of the license under which I was granted any rights to the PDF. Was this unethical?
- If I couldn't break DRM on my own, would it be wrong to download a copy from the internet?
- Given that I own the physical book, if my purchase hadn't included the PDF, would it be wrong to download a PDF of it?
- If I wanted to know what was in the publication before I paid a great deal of money for it, would it be wrong to download it?
- If I didn't have the money to pay for the PDF (as was often the case when I was a teenager and where I learned to crack software), would it be wrong to download for free -- but not distribute -- the PDF? How wrong?
... and so on ...
What is stealing, however, is the theft of rights perpetrated by the content industry by extending the terms of copyright so they can "own" ideas in perpetuity.
Because standard testing is only supposed to verify that you're storing the data locally. Standard testing is designed in a way that only helps "local storage" people, and alienates these users who get their information from their environment.
Open-book, open-note, and open-internet testing removes this barrier and levels the playing field for these "network storage" learners.
Or am I completely out of touch?
To me, it's a form of duress.
As a child I was not allowed to use calculators in math - for the reason that "what if your computer breaks?" - well, it never broke. It was a stupid rule, especially after 4th grade.
Computer access will always be available, so why ban it during testing?
Culturally we just feel significantly less guilty about it. Doesn't mean we aren't aware that it's wrong. Chinese people are fully aware of what's right and wrong and we choose to deliberately cheat. Whether that's a cultural tendency or genetic one is another story.
What?! Genetic?! There is very little, if any evidence to support the idea that genetics has anything to do with the Chinese propensity to cheat...
I only speak from personal experience. I know many chinese people both born here and born abroad. I am also chinese and born in the united states. I am telling you, honestly, from a purely anecdotal standpoint: I think there's a chance it's genetic.
Edit: Just to keep things from getting out of hand, and more balanced I want to state this fact: Statistically, it is far more likely for a serial killer to be a white caucasian male then it is for a serial killer to be of any other race. Do I think this is a cultural thing? No. I'm leaning towards genetics. But that's a purely anecdotal opinion as there's no evidence pointing in either direction.
It's very possible for many differences in behavior between races to be genetic in origin, in fact it's the more logical hypothesis versus the alternative which states genetics doesn't influence behavioral differences between races.
Think about it. If genetics influences physical traits from height, skin color, facial features, and even athleticism, what black magic in this world makes it so that genetics doesn't even touch behavior or intelligence?
>It's hard to imagine a protein causing such complex differences in behavior while affecting nothing else.
It's impossible to logically deduce a conclusion from the bottom up. We simply currently don't have enough knowledge to know how proteins scaffold the entire human neural network. With highly limited knowledge, we can only look at the problem from the top down. That being: genetics is known to influence physical traits, therefore it is logical to conclude that it also influences mental traits.
Because we have no evidence that such is the case on a culturally grouping level.
There is no genetic concept of "Chinese". It literally doesn't exist.
You're making this about the possibility of genetics impacting behavior, when the real issue is you thinking cultural boundaries exist in genetics. They don't.
If you go low enough on any topic the boundaries between categories become vague and the definitions become extremely complex. It's very hard to define what a human is in terms of atoms. The same goes for race, it's very hard to define, at the genetic level what is chinese, and what is not, but the category and boundary exists at all levels, and we can't ignore it.
I've heard of your argument before. They say that the delta in genetic differences between two people of different races is the same as the delta of two people, of the same race, therefore race doesn't exist. This argument is flawed. I believe the "genetic" definition of race is immensely more complex than simply the delta of genetic differences. Here's a more accurate definition: People of the same race have a higher probability of sharing certain genetic traits.
So let me redefine my argument in way you can understand. The people who we label as "chinese" who share similar physical/genetic traits, I believe will be more likely to also share a behavioral genetic trait that makes them more likely to cheat.
Common misconception that they do, but there is very little genetic consistency across cultural boundaries, and when such a thing does exist, it's quite noteworthy.
This statement is utterly and completely incorrect. It is a common myth in the social sciences.
Please read: http://en.wikipedia.org/wiki/Race_and_genetics#Population_ge...
first paragraph from above page: "The relationship between race and genetics is relevant to the controversy concerning race. In everyday life many societies classify populations into groups based on phenotypical traits and impressions of probable geographic ancestry and socio-economic status - these are the groups we tend to call "races". Because the patterns of variation of human genetic traits are clinal, with a gradual change in trait frequency between population clusters, it is possible to statistically correlate clusters of physical traits with individual geographic ancestry. The frequencies of alleles tend to form clusters where populations live closely together and interact over periods of time. This is due to endogamy within kin groups and lineages or national, cultural or linguistic boundaries. This causes genetic clusters to correlate statistically with population groups when a number of alleles are evaluated. Different clines align around the different centers, resulting in more complex variations than those observed comparing continental groups."
In short it's saying genetic traits can be statistically correlated with population groups (race) but variations of traits that are different within population groups can actually be more complex than those observed when compared with people outside of their race.
This is literally exactly my argument. Supported by wikipedia at the very least.
What we "tend to call" race is not defined, despite this wiki page's attempt to do so.
It isn't. The concept of "race" is not rigorously defined.
http://en.wikipedia.org/wiki/Life#Definitions
A word not having a rigorous definition means it cannot be discussed scientifically. Hence the actual problem of studying the existence of life, e.g. is a virus alive?
http://www.thesaurus.com/browse/unequivocal/4
>A word not having a rigorous definition means it cannot be discussed scientifically.
Life is discussed scientifically in many contexts yet it is not unequivocally or rigorously defined. In fact there's an entire field based on the study of life. It's called biology, aka the study of life. If a scientific field can stem from a word that does not have a rigorous or unequivocal definition, then it can be discussed scientifically.
I'm getting pretty tired too. You choose not to accept the facts even when a scientific description proving my point is thrown in your face. Ideas need evidence for support, you have presented me with ideas, but no evidence.
The folks in the field are in agreement with me, (see the old wikipedia link I sent you). You got nothing, only empty claims.
Suppose instead that a few traits like: - Social Intelligence - Impulse Control - Desire for Retribution
have normal distribution with slightly different peaks in different populations.
A population with high social intelligence and low desire for revenge might tolerate more cheating(since people only cheat when they can get away with it) than one with low impulse control and high desire for revenge, where cheating might spark a shootout.
You probably don't have to move those distributions much to start seeing pretty large changes in things like mass shooting demographics, or who cheats.
This is really offtopic, but I think you're wrong. There are two ways to do such research - the proper one and the racist one.
The proper one is to link specific genes to specific behavior. This presumes that you screen individuals in the study for those genes. Then you are not being racist, because existence of those genes is rarely 100% coinciding with culture or skin color. Research like that is quite common, if there is a reason to think there is indeed a link (for example there is a genetic study of above average intelligence in Ashkenazy Jews, they were extremely homogenous group and yet they don't all share the genes that has been shown to have the link).
The racist way is to match skin color or culture to behavior. This is scientifically useless, because you don't show any actual genetic link, and it only serves for stirring racial hate (you could pick any indirect attribute from many, like eye color or facial hair, so why pick skin color?). Such studies are rightfully being rejected by real scientists.
I personally got bit by this in my work. This is exactly why I'm never ever going to work with a Chinese again in my life time. I should also tell about this to people I care about.
Many tests verify a skill rather than knowledge. If you're cheating then you're masking your lack of skill by using another person's skill as a crutch. If you're later given the relevant qualifications and end up in a situation where other people expect to be able to rely on you having the said skill then you're likely to fail these people. Bad.
In today's world, remembering specific pieces of information locally is less of a concern, I agree, but for basics it still applies: you may be able to learn it from colleagues or internet when you need it, but there is still a limit on your "input bandwidth".
[^] With teachers like that, it isn't surprising that South Africa is so poorly ranked at maths.
http://www.washingtonpost.com/blogs/answer-sheet/wp/2015/03/...
This same thing also continues and becomes ingrained in academia:
http://www.economist.com/news/china/21586845-flawed-system-j...
I'm not sure if it's a cultural thing as you say or what, but I have heard of universities just assuming all numbers from Chinese national applicants have been doctored.
That's not what that article claims. You cannot conflate poor grades and cheating. The article intentionally misleads by just listing % for all reasons that Chinese student are expelled and then hints (without outright saying) the reason must be cheating, it couldn't be for example poor English skills? Or a ton of other problems international students might have (e.g. cultural adaption)?
I don't know if Chinese students cheat more (I don't have that data, and the article doesn't supply it). But I do know that Americans spend a lot of time writing anti-Chinese stuff and it is getting pretty pathetic.
>Demand for an overseas education has spawned a cottage industry of businesses in China that help students prepare their applications. The industry is poorly regulated and fraud is rampant. According to Zinch China, an education consulting company, 90 percent of Chinese applicants submit fake recommendations, 70 percent have other people write their essays, 50 percent have forged high school transcripts, and 10 percent list academic awards and other achievements they did not receive. As a result, many students arrive in the U.S. and find that their English isn’t good enough to follow lectures or write papers.
EDIT: The white paper linked in the WSJ article leads to a 404, but here is the link: http://www.wholeren.com/wp-content/uploads/2015/06/2015-Whit...
Google Translate provides the following:
>In the survey, we found that the main reason was expelled classified into the following nine kinds: poor academic performance (57.56%), academic dishonesty (22.98%), attendance problems (9.67%), misconduct (3.87%), international student identity (2.96%), in violation of the law (1.93%), psychological problems (0.57%), financial problems (0.23%) and Shen learn material legacy issues (0.23%). Some were expelled from reason There is a causal relationship between the links. But the statistics, we tend to count the cause and expelled most direct contact.
How did they even get admitted into university in the first place in that case? TOEFL cheating.
Anecdotally speaking, a lot of international Chinese students cheat blatantly - i.e. comparing answers with the person beside them at the end of the exam when people are turning in (and often get caught doing so).
Culture is not just cuisine and costumes -- it is also how one interacts with the world and others in one's society. There's a certain arrogance in assuming that all people in the world must obviously approach things in the same way that you do.
(Bear in mind that one can not have both "There's a lot of diversity in the world, and that's a good thing, and we should seek more diversity" and "All cultures are precisely the same". Pick at most one.)
Except when it comes to stealing land, right?
Not for nothing, but I'd actually agree with that conclusion, based on your example.
If a culture promotes fast food to the point where competitions related to the rapid consumption of the fast food causes a death, I'd consider it a symptom of the larger "fast food culture" problem for that society.
A Japanese man who wins predominately USAian eating contests does support the statement "Competitive eating is popular in Japan".
There is a quote that comes from sports but I think is more apt for business, "If you're not cheating, you're not trying" and ties well with "It's easier to ask for forgiveness than permission". This is a competition so they cheated and lost, but their gains from the work itself will/should pay dividends in the real world.
Clever would be finding a loophole in the rules, a way to run your tests without it getting counted as task submission, make it look like a bug or something like that.
There was a story about Indian students' rampant cheating at US colleges, that was presumably flag-killed off HN despite the horror stories that were emerging from academics. I've read about Australian universities basically selling degrees to foreigners who can't speak English and get admitted fraudulently. There is a currrent story about fraudulent admissions to US colleges by Chinese students as a result of massive SAT cheating. I'm not picking on Asians: Switzerland seems to have its fair share of scoundrels, as the FIFA scandal reminds us.
We need to acknowledge that gloablization sometimes brings unwanted side-effects and deal with these head-on.
Educational institutions, however, have totally abandoned this obligation. Shocking cheating behavior will merit just a note in someone's record, if administrators are even willing to take it that far. Cheating isn't publicized, shamed, and punished in the way necessary to have any impact on the cheaters' values. Instead, everyone gets to save face.
If you're worried about breakdown in the face of loss of trust, then these sorts of solutions seem like very important things to be looking at. Do we have any good ideas about how to create trust at the institutional level?
I agree with your concerns. I think they are representative of a broader theme - the acceleration of technical change seems like it will eventually (if it hasn't already) bring us to a point where the rate of cultural change is too slow to catch up to properly adapt to what is now possible.
In another course, the professor and assistants would view videotapes of the lecture to catch students who were taking quizzes for other students. Those students were dealt with as well. The XF grade stays on the transcript forever.
It may be one data point, but there is at least one university serious about cheating.
I would like to generalize that to all American universities, but I don't have enough data.
They'll be the physicians treating you and your family. Or the programmer conducting code reviews on the critical aviation software.
Given individuals are (unfortunately) in this position in many professions, we need academic rigor (until we get AI to make better decisions than the average driver/doctor/programmer)
You are theoretically there to learn your field, not to learn how to work "collaboratively", which is a vague enough description of cheating that it could be used to justify anything. Hell, if I rob a bank am I not just "sharing"? This whole concept of "theft" in finance is so outdated!
Also, it's obvious you are mentally disabled. Otherwise you wouldn't disagree with me, because I'm objectively right. You should go back inside your padded cell and take your medication. /s
Using Google's API for your own competitive product API is different. Google has not set a standard for map APIs. At least not officially. It might be THE standard for maps, but that is neither here nor there. At this point you could call it cheating. They took intellectual property and appropriated it for a clone. Plus side, we now have an implied standard. Downside, theft of intellectual property.
1. Google Play Services APIs: These are optional, proprietary APIs. You don't have to use them, and you have to treat them as optional if you want to operate across Google, Amazon, and other AOSP-derived Androids. What's the issue here? They're like any other proprietary API.
2. "Breaking" the "Java standard:" Google "appropriated," via a permissive open source license, an open source implementation of some Java base classes, and the (not protectable) syntax of the Java language. They added support for their own remote API feature and a bunch of APIs, mainly to make a usable UI system. The result runs on a runtime Google devised, using a bytecode Google devised. Where is the intellectual property theft?
For this test, they should have had an automated way to enforce that rule, if possible.
Look at the actual levels of piracy in the US versus China. In the US, you will get some people who say they are worried it might be wrong or that they will get caught. But most people will say something like 'watching this free stream doesn't hurt anyone'.
In the US, we have no problem with high end shoes that look similar to one another, or low-end similar to high-end. But there is a subtle distinction -- there must be some way to claim that this is not copying the other. But maybe it more often comes down to taking a different attitude towards the same thing. E.g. "cheap Chinese knock-offs" versus "inexpensive sensible alternative to overpriced designer brand".
Its also perfectly fine in general for companies to copy a successful business model. But we insist that there must be some distinction. However the difference between these companies may only be slightly greater or mainly surface-level, and so when you get down to the fine analysis, I thinj the real difference between the cultures is smaller than people want to admit.
How does it work? You submit your classifier to some server and it is run against what? The data set that determines your final score - hopefully not.
See my comment & links at the bottom of this page. For some reason HN always puts my comments near the bottom, even when they're getting many upvotes. (I'm guessing that HN has a comment-editing penalty from editing my comments too many times, but I'm not sure, or maybe it's from times I must have insulted the mods here.)
Edit:
I constantly get "submitting too fast", so I'll reply here.
Thanks for the info kefka. Yeah, I'll happily agree with that linked comment, that the childish crude moderation techniques used here have been hurting a once great community. It was the exact same scenario for me as for that guy. Seems I got rank-banned immediately after confronting dang about the trends of excessive downvoting and increasing side-project criticizing on HN.
I think it's because you don't have many upvoted comments in total (maybe?). I upvoted you here, though.)
A mod did that to you. See this: https://news.ycombinator.com/item?id=9165780
The technique is actually pretty fascinating. This is something that's been well understood by the cryptography community for decades, but is somehow just recently being fully appreciated by the ML community. See here:
http://blog.mrtz.org/2015/03/09/competition.html
https://www.kaggle.com/c/restaurant-revenue-prediction/forum...
Summary -
Submitting guesses to a system that gives you back scores for your guesses, will quickly leak out enough information that you can reverse engineer a huge number of hidden numbers/labels in surprisingly few iterations, e.g. 700 iterations to covertly extract 10,000+ real numbers with high precision. This surprisingly rapid convergence is a bit reminiscent of the birthday paradox.
Further, this not only lets you win the against the "test" dataset, as apposed to the final "validation" set, but this allows you to significantly increase the data available to you to train your model on, since now you can train your model against both the "test" and "training" datasets.
Layman summary -
ML breaks datasets into 3 partitions "test", "train", and "validation". In cases where they're evenly split, this technique can double the training data you have access to, which is a massive advantage in ML competitions where scores differ by tiny amounts.
Moral judgement -
My opinion, this moral argument is misdirecting the attention from where it needs to be. Yes, it's bad what occurred here. But at this point, in 2015, and with tools readily available to crack this problem effortlessly, it's inexcusable for contests to allow so many scoring reports against their validation sets anymore. It's no longer a question of whether contestants will do it, but how many of them will. We'd might as well just let people self-report their scores on an honor system, if we're going to be this overly trusting.
Try creating a contest system like this in the cryptography field any time in the past 3 decades and you'd be insulted and laughed out. Allowing so many scoring reports against the validation set is fundamentally flawed. The only solution is to globally limit calls to the scoring api.
Another proposed solution -
Allowing everyone to see everyone else's guesses & resulting scores against the "test" set, so that everyone is on equal ground for reverse engineering the "test" set, and then globally limiting the number of scoring attempts so that the test set isn't reverse engineered too significantly.
Overfitting the "validation" set actually is not a problem either way, because none of these contests are dumb enough to let anyone score against the validation set at all until the contest submission deadline is over.
The "right way" to do it is test once only per model/paper. If you are interested there are a huge number of sneaky ways overfitting can happen in ML [1]. Also interesting that you too see crypto and ML as related - I see them as opposites of the same coin. One tries to pull signal out of noise, the other tries to bury the signal in noise... but special noise.
HOWEVER, the goal of these contests should be to promote the most accurate and powerful image recognition algorithms that will transform the world as we know it. Limiting access to training data makes it more difficult to test changes to an algorithm. These rules do not make sense to me, and I would advocate against them.
Second of all, cross validation : http://en.wikipedia.org/wiki/Cross-validation_%28statistics%.... You don't HAVE to submit to the test set to get a feel of how well your model is performing. On datasets this large, cross validation should be effective, if more time consuming method (unless your model is extremely unstable).
Given enough attempts at the test set it's very easy to overfit your model to that test set, meaning that although the accuracy looks higher, it would generalize worse on a different subset of the data.
>"promote the most accurate and powerful image recognition algorithms that will transform the world as we know it"
as you say, then running hundreds of attempts at this one data set to do better at this specific competition won't foster that goal.
The first known use of the word "astir" is from 1765, says Merriam-Webster at http://www.merriam-webster.com/dictionary/astir . This is what I would expect if "stir" was a word, "a-" was a possible mechanism to create new words, and eventually people started to use "astir.
Etymonline gives a first known use as 1823: http://www.etymonline.com/index.php?allowed_in_frame=0&searc... :
> "up and about," 1823, from phrase on the stir, or from Scottish asteer; from stir. Old English had astyrian, which yielded Middle English ben astired "be stirred up, excited, aroused."
The root "stir" has a longer heritage. Etymonline at http://www.etymonline.com/index.php?term=stir&allowed_in_fra... says:
> Old English styrian "to stir, move; rouse, agitate, incite, urge" (transitive and intransitive), from Proto-Germanic * sturjan (cognates: Middle Dutch stoeren, Dutch storen "to disturb," Old High German storan "to scatter, destroy," German stören "to disturb"), from PIE * (s)twer- (1) "to turn, whirl" (see storm (n.)).
Hindi is another descendant of Proto Indo-European, so that may be where there's a connection to "asthir". However, do bear in mind that the surface similarity is false - astyrian would be linguistically closer to the Hindi than astir.
But it is not uncommon for word meanings to change in such a way that they take on a meaning exactly opposite of what they used to mean (see for example the meaning of the word nice: http://www.etymonline.com/index.php?term=nice&allowed_in_fra...). I wonder if there is a name for this phenomenon.
I think that was intended to humorously imply that the link appears down when accessed from China.
Why can't the competition have the same test data each week across all participants? So that no matter how many accounts you create, you will train with the same images everyone else gets to train with.
Having never been employed but having coded for 8, I wish I could get myself educated in this field.
As for turning current contests into an exercise in overfitting the "test" set - we already reached that point long ago. Test vs validation scores often diverge wildly in these contests.
Edit - Replying to arnsholt:
Completely true. The huge problem I see, is that all the classic NLP tagging corpuses are created from the very narrow domain of news articles, and a few good corpuses now appearing for biology texts, and that's about it. Want to do, e.g. NER for product reviews or chat logs? - Incredibly bad results. There's a huge corpus problem in NLP today.
I'm not familiar with computer vision, but in NLP taggers are hovering around human-level performance, and parsers are quickly approaching that level. But if you take a state-of-the-art system and test it on a slightly different corpus (even something as simple as text from the same newspaper, but a year later!) performance drops by a lot.
They are not improving the algorithm for the interest of humanity. They submitted and obtained test results much more frequently than allowed. With that information, they can tune parameters to more closely fit the test data.
Basically it is like scoring higher on an exam where some of the questions have been leaked. It does not suggest you now have a better understanding of the subject.
It is NOT like looking at leaked answers. It's like re-reading the textbook more than everyone else. Definitely an unfair advantage however.