make a request on the scholar subreddit
1,376 karma · joined May 12, 2015
Machine Intelligence, functional programming, photogrammetry, and Machine creativity.
Currently : Transferring learning between modalities in robotic neural nets.
[deepnet] [\@] jmhz [\.] net
tl;dr Seth Bling directly references NEAT and his links[Show More [1]] link to 2 Togelius papers[13][14] among many others !
To wit, in OP, Togelius mentions Seth Bling's MarI/O[1] as lacking scholarship because he didn't mention Togelius:
" seen as research, it is atrocious because of the complete lack of scholarship. The guy didn't even know he was reinventing the wheel, and didn't care to look it up. "
Rather disdaignfully (sadly/ ironically/ poigniantly) Togelius refers to Seth not by name but only as " some guy ".
Yet Seth Bling[2] in his short video, mentions and explains NEAT and directly links Ken Stanley's and Risto Miikkulainen's foundational NEAT paper[10] at U.texas which Togelius builds on [hint: click SHOW MORE under Seth's video]
And much more, i.e.: ---- the innovative toolchain Seth uses: Bizhawk to script SNES emulators in Lua [12]!
---- Seth links the wikipedia page on Neuroevolution[11] which directly references 2 of Togelius's own papers[13][14]
Seth's [Show More] Links on the MarI/O video[1] are very very comprehensive and (IMHO) indeed a great example of scholarship !
Togelius is a top flight scholar, and publishes in the field of AI game players and level generators[15] so I hope he will welcome some random tinkerer pointing out his error.
[full disclosure: I am a Togelius fan]
To give Togelius his due he does at least link to Seth's MarI/O video.
Perhaps Togelius failed to click / read the Show More links ? The charitable benefit of doubt principle is accidental oversight.
Seth Bling is primarily a Twitch Streamer[2] and Speed Runner[3] who often popularises complex programming[4], math and academic ideas[5] to a primarily young audience using Mario and Minecraft[6] and takes requests via his subreddit[7]
Seth's very popular Youtube video MarI/O which uses NEAT (Neuro Evolution of Augmenting Topologies] to learn and play Super Mario World, has led to a huge number of streamers on twitch playing other MarI/O levels[16], using RNNs, LSTMs, and other SMW levels and SNES games, like RNN multiplayer x4 Mario Karts[17].
The OP article is great, academic rigor and bibliographic scholarship might sometimes differentiate the full time university scholar from the amateur researcher, enthusiast or tinkerer.
Seth and his many followers may not be 'scholars' as in academic research but it is an amateur community of mostly youngsters doing their own research by having fun and sharing ideas, mostly undocumented outside their own transient twitch streams.
Compare this to the well respected amateur astronomy community and perhaps a responsible academic approach is to try to reach out and tap in rather than adding to the distance.
By his own OP standards of Togelius is lacking in that hasn't discovered/ mentioned the work done by the Twitch/ Bizhawk community - it is not entirely undocumented [16].
Togelius your AI Mario competition[8] is legendary in academic AI circles but not necessarily widely known by youngsters playing Mario ( and to be fair AFAIK you don't publish in their forums like twitch) although the code Infinite Mario[9] you base your work on is by Marcus Persson[10].
Though Persson is better known as Notch [10] ( author of Minecraft ) who is more famous than perhaps even Alan Turing , Yann le Cun, Jurgen Schmidhuber, or Geoff Hinton to the under 16 crowd.
Perhaps Togelius could mashup some of his work and do a crossover episode with Seth or relaunch the Infinite Mario competition[8] / benchmarks to this new younger crowd evolving Neural Topologies to play Mario games - or even help document some of their fun.
As public servants funded by tax dollars it is perhaps the responsibility of the academic to reach out rather than blaming fun loving amateurs popularising and doing research / 'tinkering' for not putting in the due diligence of scrupulously mentioning every source.
[IMHO] I agree amateurs don't always do full and proper scholarship and links are food for hungry minds - but they aren't publicly funded researchers but amateurs working for love of the subject in their spare time, for free, or for streaming donations and youtube hits, using transitory, live media and whose audience must be constantly engaged and may have shorter attention spans than the average academic.
[full disclosure I am an amatuer neural net nerd]
[1]Seth Bling's MarI/O https://www.youtube.com/watch?v=qv6UVOQ0F44
[2]https://www.twitch.tv/sethbling
[3]https://www.youtube.com/watch?v=-spFoon7klA sub 1 min SMW via credits warp
[4]Bling's arranging shells in SMW to human inject Flappy Bird code into SNES https://www.twitch.tv/videos/57032858
[4]Bling's Atari 2600 emulator built in Minecraft https://www.youtube.com/watch?v=5nViIUfDMJg
[5]Seth Bling's RNN Mario Cart race https://www.youtube.com/watch?v=Ipi40cb_RsI
[6]Bling's Minecraft Redstone tutorials https://www.youtube.com/watch?v=DzSpuMDtyUU&list=PL2Qvl4gaBg...
[7]https://sethbling.reddit.com
[8]Togelius 2009 Infinite Mario AI competition http://julian.togelius.com/mariocompetition2009/
[9]https://web.archive.org/web/20080423023424/http://www.mojang...
[10]Stanley Miikkulainen NEAT paper http://nn.cs.utexas.edu/?stanley:ec02
[11] https://en.wikipedia.org/wiki/Neuroevolution
[12] Scripting Lua in Bizhawk Emulator http://tasvideos.org/Bizhawk/LuaFunctions.html
[13] Risi, Sebastian; Togelius, Julian (2017). "Neuroevolution in Games: State of the Art and Open Challenges" (PDF). IEEE Transactions on Computational Intelligence and AI in Games.https://arxiv.org/pdf/1410.7326v3.pdf
[14] Togelius, Julian; Schaul, Tom; Schmidhuber, Jurgen; Gomez, Faustino (2008), "Countering poisonous inputs with memetic neuroevolution" (PDF), Parallel Problem Solving from Nature https://www.academia.edu/download/30945872/poison.pdf
[15] Togelius' Game AI book http://gameaibook.org/
[16] https://clips.twitch.tv/CourteousEmpathicTruffleCeilingCat
[16] https://thenextweb.com/artificial-intelligence/2018/01/03/th...
"Udacity Universe offers virtually limitless opportunities to interact, collaborate, and problem solve, and is exclusive to students of our Flying Car and Self-Driving Car Engineer Nanodegree programs."
For instance finding out the address of MOMA in the first place may require asking someone or googling it.
The stronger version is that some interesting ideas can only come about by dialogue between specialists.
Sometimes a dilemma for scientists is whether to share and benefit from input but then perhaps give away the key insight to others. Q.v. Wilkes showing Crick Rosalind Franklin's DNA crystallograph.
Proper courtesy, accreditation and citation can in some cases avoid this - yet Wilkes told no-one he was working on Fermat's last theorem for many years until he thought he had a solution.
The OP is by Risto Miikkulainen who was Stanley's collaborator on NEAT and should not be summarily dismissed.
According to this BBC research http://www.bbc.co.uk/news/health-38800977
Guard the paths; protect the weak.
Teach what you know.
This journey, your journey, the words you wrote they will be questions asked silently elsewhere by others.
In asking you are already brave.
Write about the journey as you have written this question.
This will help others and perhaps it will lead to someone suggesting something, another good heuristic for knowledge is have many other minds helping to answer your questions.
Marcus Hutchins a British security researcher registered this domain.
As the domain offswitch ping was obvious Hutchins was seemingly the only competent researcher who read the source code.
Hutchins, Britain's Saviour of the NHS[2] (Britain's National Health Service) was outed by the British tabloid press that published his home address.
Next Marcus Hutchins was arrested while attending DefCon and is currently awaiting trial in the US on unrelated charges. [3]
Wannacry had spread throughout the NHS and was preventing actual life saving operations, it spread on some Microsoft XP and mostly unpatched 7 systems, from the audit:
"NHS Digital told us that all organisations infected by WannaCry shared the same vulnerability and could have taken relatively simple action to protect themselves .
... NHS Digital told us that the majority of NHS devices infected were unpatched but on supported Microsoft Windows 7 operating systems. Unsupported devices (those on XP) were in the minority of identified issues. NHS Digital has also confirmed that the ransomware spread via the internet, including through the N3 network (the broadband network connecting all NHS sites in England),"[4] NHS Digital Audit
[1] https://whoapi.com/blog/3079/what-is-the-domain-name-that-st...
[2] http://www.independent.co.uk/news/uk/home-news/nhs-cyber-att...
[3] http://www.independent.co.uk/news/uk/home-news/marcus-hutchi...
[3] https://www.nao.org.uk/wp-content/uploads/2017/10/Investigat...
From a comedic dystopian science fiction perspective this coincides the politics of underfunding the NHS, institutional costs of running Microsoft's patches on vital life-saving equipment, the US intelligence services offensive rather than defensive playbook, and an "unlikely" hero.
Might make a great novel except if ludicrously we are about to live it as a pretext.
This is a brilliant idea and it would like to see it.
If the Socratic method get stuck, ask a different question.
Can questions so implicitly contain the answer that one can practically make a statement with a question ?
Those in charge ultimately bear the responsibility.
Not criticising leaders is the road to ruin, and senseless jingoism.
National humility trumps national pride as a useful virtue.
Don't forget Stallman saw the destruction of his beloved Hacker Culture of Sharing and Academic Discourse eroded and broken by the LISP machine debacle and the change to a closed anti-sharing approach of UNIX by the newly privatised Bell in the 1980s.
And it was a proprietary printer driver which initiated his epiphany that in a world governed by proprietary software the user would have no freedom or a freedom that could be taken from them so no right to freedom. RMS could easily have made the modification to the driver he wanted but he was not allowed the source code.
From this he invented his four essential freedoms [1] manifesto and the GPL which allowed developers to develop with out fear of the closing of their contributions behind paywalls.
In the early 1990s the Linux kernel flourished with its GPL license ensuring it remained a bastion of freedom.
As Stallman advocates for a future where users have maximum freedom then proprietary software is opposed to that.
If I use Photoshop over GIMP and write tutorials and encourage others to use Photoshop then the world is a little less free for those users in the sense of the four essential freedoms [1]
By this definition proprietary software is opposed to Stallman's vision and insiduously so.
To oppose the freedom of others is in RMS's eyes not a good act, thus it can be rightly called evil in that context and IMHO he is not wrong in this.
Proprietary software 'may' not steal your freedom today but it can lock away your contributions and data tomorrow and one has no recourse - Stallman having lived through this wishes to oppose it and fight for the right to Freedom for us all and forever.
His is a glorious vision and he is not wrong, without free software competing with it, proprietary software would likely offer much less freedom and can and has taken what freedom it offered away !
We will live in an increasingly computerised and robotised society. We must choose : will we only have tenuous freedom exchanged for conveniences and mind-candy, rescindable freedoms only for some given by the whim of corporations; or will we work together on GNU and other Free projects and live as Free peoples of the solar system ?
Free as in Liberty, Egality, & Community.
[1] https://www.gnu.org/philosophy/free-sw.en.html
Oh ye of little faith, history has so far found that :
Shannon places Isaac Newton & Alan Turing above the curve, so it a very high bar.
This is Shannon writing at age 36, still at Bell labs producing practical work for the military, often classified and mostly on crypto.
His seminal memorandum 'A practical Theory of Communication' had begun circulating only 4 years earlier and it's full impact was not yet apparent.
Per your useful suggestion I made a subreddit as an online forum/ discussion group for participants ( in the 2017 Berkeley DeepRLBootcamp ).
Great, I am flying over from Scotland for the Berkeley DeepRL bootcamp.
I too am a big fan of Abeel's lab. His papers with Levine and Finn - very far sighted stuff. I have been working through the CS294-112 Deep RL course materials [1] recently.
A bio might be a touch off topic for this thread, very happy to enthuse about Neural Nets, Reinforcement Learning and Robots though - I can be emailed at deepnet at jmhz dot net
[1] https://www.youtube.com/watch?v=f4gKhK8Q6mY&list=PLkFD6_40KJ...
The above links to Pieter Abeel's CS294-112 lecture on Adversarial Nets and his Thesis on Helicopter Contollers with Andrew Ng.
I am not saying that at all. Daniel Dennet is better placed to speculate on matters like that.
[/edit] perhaps there is some common ground. Brooks' Behaviour-Based position, though useful at the time, threw out the symbolic AI with the bathwater of computational complexity that robotics was quagmired in then.
Certainly Brooks ran into Moravecs paradox, that walking is the hardest form of thinking and playing chess and planning less so. And so Brooksian embodied robotics never got to the symbolic level.
Contrary to your anthropocentric claim, Brooks (and followers such as Tilden) were biomimetically inspired by the intelligence of insect locomotion and 'intelligent' engineering like the underslung chassy of spiders and beetle.
I do agree with Brooks that AI will not emerge in simulation due to the lack of real stochasticity and resolution.
[/edit] I will temper my stridency and say that it is less likely, my initial absolutist position was merely excitement and really simulation has a big role to play as imagination and plan testing. We live in a world of perception and as [Richard] Gregory says "perception is only a hypothesis, based on our prior knowledge and updated by reality".
For instance the Atari benchmarks appears random but the randomness is Mersenne Twisters with few seeds and most expert (human) arcade players learn the patterns.
That is not to say that Mnih wt al.s Nature paper wasn't groundbreaking and their Deep-Q RL is certainly insightful, promising and useful but (IMHO) the AI 'stack' and LeCun's Predictive Learning will not come about outside of a real robot acting in the world and using raw sensor data to power actuators directly.
My position is not entirely Brooksian. I follow Esther Thelen in the view that a being ( a brain and a body ) needs to learn to be by being and moving and feeling and touching.
I do apologise if my brevity and excitement appears rude, AI could well emerge in computers running simulations and in fact I think that simulation can play the role of planning and imagination but it will need to be grounded in a real physical agent - a hybrid approach will bear the most fruit.
Rodney Brooks famously condemned simulation and games positing that AI can only develop in an embodied agents.
Robotics with deepnets is the edge and Berkeley's Abeel, Levine and Finn are right on it (IMHO). This paper puts it all in valuable context.
Anyone else going to the DeepRL Bootcamp at Berkeley in August ?
Though (IMHO) MCTS is better characterised as evaluating moves rather than exploring plans.
The MCTS only explores the moves in order of likelyhood using the most basic of heuristics, random playout.
The Net outputs likely moves based only the current board position, it formulates no strategy.
No state is stored across moves - each play is independent, relying only on the current board position.
I still don't see anything anywhere in AlphaGo that is a plan, trajectory or strategy.
Neither is there an evaluation of the opponent nor any attempt to outwit them.
That it performs so astonishingly well without a plan is very very interesting and should perhaps give us pause - is planning a hubris ? Do we undervalue our use of heuristics in our own behaviour ?
I was under the impression that AlphaGo makes no plan but responds to the current board state with expert move probabilites that prunes MCTS random playouts.
There is no plan (AFAIK) or strategy in the AlphaGo papers so I find this statement that AlphaGo is an imaginative planner quite curious.
Perhaps someone can reconcile these statements or correct my knowledge of AlphaGo ?
Very interesting papers, it will be nice to see the imagination encoder methods applied to highly stochastic enviroments or indeed a robot in the real world.
E-commerce has enabled many small 'boutique' or 'mom-and-pop' style shops to survive in locations without sufficient footfall.
The combination of small business bricks and mortar engendering trust with e-commerce supplementing footfall and enabling the tourist dollar to be spent after the holiday has helped many small town artisan and craft stores.
[1] 1st Q 2017 ecommerce US census report https://www.census.gov/retail/mrts/www/data/pdf/ec_current.p...
The other widely reported vector algebras in a semantic space were discovered by Mikolov et al when producing ~300 dimensional vectors for a billion word Wikipedia corpus.
If one performs vector algebra and ~= is near by cosine distance then using Mikolov's Vectors[3].
King - Man + Woman ~= Queen
France - Paris + Gernmany ~= Berlin
Karpathy shows trained nets can be assembled like lego across modalities, slice off the classifier to reveal the rich semantic 'thought vector' layer of an Imagenet trained Alexnet, plug in a RNN sentence generator using word2vec and ( some over simplification ... ) you get a convincing image captioner [6].
The thought vectors are akin to high level representations of the world and can cross modalities . Text to Images using thought Vectors ( from hnnews discussion [7] )
So the vectors of though are in some way a an AI mentalese or encoding of a symbolic representation of the world derived from the data and can ( again drastic over simplification ) transfer modalities and even between previously unlinked languages [8]
Also see Anything2Vec https://gab41.lab41.org/anything2vec-e99ec0dc186
[1] https://youtu.be/izrG86jycck?t=25m58s
[2] The paper Geoff Hinton is reffering to : Sequence to Sequence Learning with Neural Networks by Ilya Sutskever, Oriol Vinyals, Quoc V. Le https://arxiv.org/abs/1409.3215
[3] Efficient Estimation of Word Representations in Vector Space by Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean https://arxiv.org/abs/1301.3781
[4] Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks Alec Radford, Luke Metz, Soumith Chintala https://arxiv.org/abs/1511.06434
[5] https://code.facebook.com/posts/1587249151575490/a-path-to-u...
[6] SF Machine Learning: Automated Image Captioning with ConvNets and Recurrent Nets by Karpathy https://youtu.be/ZkY7fAoaNcg?t=38m31s
[7] https://news.ycombinator.com/item?id=12366684
[8] https://github.com/Babylonpartners/fastText_multilingual
Over constraining your hiring, ( i.e. the aspect of "culture-fit" that is antonymical to diversity ) can lead an organisation to blindness to the true state of things ( i.e. not having diverse perspectives on customers due to a lack of diversity in employee backgrounds ).
This is why diversity is not just about fairness but also highly pragmatic.
Diversity makes for wiser corporations which makes them anti-fragile and gives them new edges in competition.
It is also why the subtle method of subverting democracy outlined in Chomsky's Manufacturing Consent as a practice to create an organisation ( or newspaper ) that outputs a world view concurrent with ones own makes for fragile democracies and administrations.
That it can happen "unconsciously" because it is easier to recognise competence in those who are like you means organisations and democracies must be vigilant against entrenched political and journalistic class evolving.
In short: critics are better friends than an echo chamber.
Or contemporaneously: firing scientists does make climate change less true.
I think this is quite profound and inspiring.
Although perhaps it is only one half of it. Concretely, deep learning finds patterns, the best patterns are derived from the highest bandwidth signal, often this is the input.
Geoff Hinton has argued that the task, solving the problem, is a low bandwidth signal.
Hinton aphorises [approximately] 'If you want to learn computer vision first learn to do computer graphics, i.e. a generative model.' - this is about the bandwidth of the data signal.
Hinton: [1] "Each image has much more information in it than a typical label... Each image puts a lot of constraint on the identity function. Whereas if I give you an image and a label and I try and get the right answer I don't get much constraint on the mapping from image to label. The bits of constraint on that mapping imposed by training example are just the number of bits to say what the answer is which is not very many."
Ownership of traditional mainstream media is cloistered in few hands and those hands have agendas for editorial policy and friends in high places.
Chomsky's Manufacturing consent well demonstrates how this is achieved bottom up: i.e. one isn't so much told what to think but hired because of ones views and worldview.