163 karma · joined March 30, 2014
Oh wow, didn't realize that these were multi-layer pre-trained models.
Also, started going through the QRNNs, they mention they've updated the AWD-LSTM Language model to use QRNNs, which is what your paper uses!
Curious how it'll perform compared to fasttext when used as encoding network in larger tasks. I can't help but notice the trend of going back to simpler models with smarter optimizations and regularization to achieve better results.
This is a frequent question of mine, which I ask to everyone using RNNs - what do you think of the idea that CNNs will be able to replace RNNs for sequence tasks [0]? CNNs are less computationally expensive too, so there's a definite benefit of switching to them if the performance is on par.
IRC: https://github.com/thelounge/thelounge/ (Use a lot, and it works decently well)
Cloud: https://github.com/nextcloud/server/ (Works surprisingly well, however, I've heard there are security issues)
Analytics: https://github.com/matomo-org/matomo/ (I'm comparing how much I'd lose out compared to google analytics, if I ever move away from it)
Chat: https://github.com/RocketChat/Rocket.Chat/ (never really use it, just wanted to see how it works)
Git: https://github.com/gogits/gogs (Mostly for mirroring git repos)
Browser IDE: https://icecoder.net/ (I'm able to edit the code for the bots/projects I host from the browser itself)
Calibre library front end: https://github.com/janeczku/calibre-web
Pastebin: https://github.com/LINKIWI/modern-paste (never use it)
Linkshortener: https://github.com/LINKIWI/linkr (occasional use)
Mail + mailbox + webmail: https://mailcow.email/ (Had to spend a bit more time, but now it is able to deliver mails to gmail/outlook/icloud without issues. I use it as a mailing solution for all the selfhosted projects which need smtp)
Online Markdown editor: https://github.com/joemccann/dillinger (Used quite a few times)
Minecraft server
Music streaming: https://github.com/phanan/koel/ (I'm not a native english speaker, so quite a few songs I listen to aren't on spotify)
Neo4j, mongodb, mysql and postgres: For all database needs.
Server monitoring: https://github.com/firehol/netdata
Photos: https://github.com/Chevereto/Chevereto-Free (I use it quite a lot to host images I'd have used imgur instead)
R Studio server, and Jupyter notebooks: For work/hobby programming
I've had a lot of help from https://selfhosted.libhunt.com/ and https://www.reddit.com/r/selfhosted/
I"ve had to move from one VPS provider to another while testing, so I have a handy guide for setting up quick here: https://github.com/itsmehemant123/basic-vps-setup .
I have a separate guide for hosting each of the above projects behind an nginx with https, but its quite rough right now. So its private.
Yeah, I understood the intention behind that statement. Great work with the course!
While you're here, what do you think about using temporal convolution for sequence tasks? I've read a few articles, this particular one by my professor comes to mind now [0], which say CNNs could work extremely well for the tasks traditionally done with RNNs. A recent paper by the people at Google Brain [1] mentioned that their CNN with attention network beats traditional RNN approaches. More surprising is that the network is 130+ layers deep, and yet trains faster than RNNs. Do you think we can potentially switch most machine translation tasks to CNNs?
[0]: https://towardsdatascience.com/the-fall-of-rnn-lstm-2d1594c7... [1]: https://twitter.com/lmthang/status/989261575482560513
Quite a few new network architectures in this space have been updates to this model, which uses an RNN encoder and a decoder, along with attention between them and a beam search for better results.
I wouldn't call this model a solution for natural language translation, nor would anyone else. But I think fast.ai meant that they're going to explain and go through this model, and how it's helped bring a new generation of models with good performance in this particular space.
You'd have to balance the input and output connection's granularity - too many would put off users, too few would make users feel restricted. If you manage to find a sweet spot, or let the user pick the level of expertise, and reveal them accordingly, it would be perfect.
I really like how good it looks, and can't wait to use it myself!
Coming to community model zoo, would it be free access to any model, and pay for the training and disk usage (floydhub like)? Or you'd go the quantopian route?
Do you have a team implementing most of the new state-of-the-art model architectures (given how fast new ones keep getting published)?
If so, I'm assuming you keep associating some types of model architectures to the type of data being input? I'm just curious how you'd pick a particular architecture.
On the other hand, AutoML comes to mind, but IMO, the biggest hurdle of AutoML, and its ilk is the massive computational infrastructure requirements.
But great job, it looks really good and seems pretty intuitive!
This github repo, along with r/selfhosted [0] and this [1] are everything you'd need.
I've been keeping track [2] of the things I've had to do to whenever I used to switch my VPS providers. Its not perfect - I don't restrict ssh, nor do I restrict the database access, but I mention that, and you can choose not to do that.
[0]: https://www.reddit.com/r/selfhosted/
However, as seen here [1], people have managed to use other GPU cases, with NVidia cards.
However,
- They have two data center locations, both in Germany, which might be good or bad depending on how view the laws.
- Their web interface looks good too, but doesn't have the features that DO provides.
- Depending on what option you pick, some have commodity hardware (No ECC RAM, i7 processors instead of server hardware).
- [Rumor]I've heard that they are very hands off, and will take down your instance much more aggressively. As someone on reddit put it, "If Disney was a cloud provider, anything Disney won't be comfortable to host, Hetzner won't."
I've been using Pytorch for quite a while, and had to use tensorflow recently. May be its my pytorch _priors_, but using tensorflow felt weird, and un-intuitive. Any good resources for pytorch folks to get into tensorflow?
I then got a usb c/thunderbolt to display port for 4k 60fps, and the issues significantly dropped, but it still occasionally happens.
The way I see it, having hooks into the engines themselves helps with what the article talks about - not needing to go through VNCs or other _glue_ to get realtime data. It could potentially send the framebuffers themselves directly from the game/simulation and tie in the actions back to the game/simulation. And using framebuffers is just one direction, we could instead stream the co-ords/the current payoff/etc.
Also, having such plugins would help with the adoption in both directions - games now have an always updating/learning AI (might need a network connection + cloud backend), and researchers can have training/testing environments.
Location: West Lafayette, IN
Remote: Yes
Willing to Relocate: Yes
Technologies: Javascript/React, Ruby on Rails, C# .Net, Node.js
Résumé/CV: https://www.linkedin.com/in/hemanth-devarapalli-40a36162
Email: hemanth.devarapalli (at) outlook
1st year Masters student at Purdue University - West Lafayette. I'm doing coursework involving Natural Language Processing, Computer Graphics.
Well, it's just a leaked image, so can't draw too many conclusions out of it I guess.