gh1

I think the idea is not to replace learning the theory or math, but rather to just postpone it. Learning the practical aspects of an engineering discipline can provide the necessary motivation to study the theory/math, and this is the oft ignored factor. I also think there is some benefit in learning a topic using the tool that you will actually use in production (if that is possible without adding unnecessary complexity in the syllabus).

I personally learned DRL from David Silver's course and Sutton & Burto back in the days. They were the only good resources around and I liked them very much. But I think that with the advent of high-level frameworks in DRL, there are better learning paths.

I do intend to teach the theory/math in a later installment of this series, but I wanted to do it by showing students how to implement the various classes of algorithms e.g. Q-learning (DQN/Rainbow), policy gradients (PPO) and model-based (AlphaZero) using RLlib. This would kill two birds with one stone: you can simultaneously pick up the theory/math and the lower level API of the tool that you will be using in the future anyway.

Depends on what you mean by faster. Do you mean "time to solution" or "time to inference"? I think there are also more factors to take into consideration when considering the merit of the method e.g. performance, robustness, ability to handle non-linearity, ability to solve the full online problem etc.

When all these factors are taken into account, I have encountered situations where Deep RL performed better.

There are also very public examples of this e.g. Google's data center cooling [0] and competitive sailing [1].

[0] https://www.technologyreview.com/2018/08/17/140987/google-ju... [1] https://www.mckinsey.com/business-functions/mckinsey-digital...

I think the levels (high, low etc.) are relevant for the Deep RL algorithm, not the environment. The lower level version of OpenAI Gym canned environments would be custom Gym environments. I don't see much reason to go any lower than that.

The situation looks different for Deep RL algorithm. You can implement them from scratch yourself using Tensorflow or any other similar library. Otherwise, you could just use a higher-level library like RLlib which implements the algorithm using modular components and exposes hyperparameters as configuration parameters.

In many real world use cases, all one needs to do is to use RLlib's implementation and then tune the hyperparameters. In that way RLlib is to Deep RL what Keras is to Deep Learning.

This course uses RLlib. Does that answer your question?

Maybe this would help you differentiate: GPT-3, DALL-E 2 etc. uses transformers, while AlphaGo, OpenAI Five etc. uses Deep Reinforcement Learning. They are not mutually exclusive, but just different things.

Good catch. I can imagine that this is annoying. I have put it in my todo.

I hope you enjoy the course over the weekend.

Thanks. I learned it from an Udemy course [0]. Took just a couple of weeks to pick up. Regarding overlays, Sketchbook supports the idea of layers. I simply put different elements in the illustration in different layers. Sketchbook gives me PSD files that can be imported in GIMP. I then export many PNG files by progressively selecting more layers in GIMP. These PNG files go into Beamer like this:

    \begin{figure}
        \includegraphics<2>[width=0.35\textwidth]{images/shop/1.png}%
        \includegraphics<3>[width=0.35\textwidth]{images/shop/2.png}%
    \end{figure}

The % sign is important and it maintains the correct positioning of the images.

[0] https://www.udemy.com/course/drawing-for-trainers-leaders-an...

My experience matches yours. Recently, I was trying to solve an optimization problem using Deep RL. As usual, I had to run many experiments over several days using various tricks and hyperparameters. Finally, it turned out something related to the symmetry of the action space made a huge difference in learning.

Anyhow, the experimentation stage requires a certain discipline and feels tedious at times. But the moment when learning takes off, it feels great, and for me personally, compensates for the tedious phase before.

It's certainly not fun for everyone, but I guess it could be fun for the target audience of the course (ML engineers/Data Scientists).

Regarding frameworks, my experience has been different. I find RLlib to be more modular and adaptable than SB3. But the learning curve is certainly steeper. The biggest differentiating factor for me is production readiness. Assuming that we are learning something in order to actually use it, I would recommend RLlib over SB3. The equation for researchers may be different though.

I am sorry about that. Unfortunately, the same thing happens in Firefox when the tracking protection is set to "strict".

This is apparently happening after Teachable updated their video player. Earlier, they used Wistia. Now they use Hotmart.

I have informed Teachable about this issue. They said they will look into it.

The current workaround would be to use Chrome or Firefox (with tracking protection set to a level below "strict").

I am glad you like it. The coding exercises don't require a GPU. Thankfully, most RL problems (and certainly the ones used in the course) require small neural nets which can be trained in reasonable time using a CPU.

Hi all, creator of PythonBooks here. PythonBooks is running since 2017. During launch, it was on the front page of HN [0]. Thanks to the HN community for showing so much interest in the website. There has been steady traffic ever since and I have tried my best to keep the website updated with the latest books. Starting this month, there's a new page which lists the best Python books published every month - based on popularity, topic and novelty. I thought that it could be an useful tool for Python developers to cut through the noise and keep abreast of the best recent books. Feedback is welcome :-)

[0] https://news.ycombinator.com/item?id=14769317

In my opinion, the best problem to solve are your own problems - both emotional and material.

Yes, different priorities lead to different results. If you are looking at life satisfaction, then 8 hours a day is claimed to be optimal [0].

[0] https://link.springer.com/article/10.1007/s12232-016-0256-1

The article claims that catching up on sleep during the weekend doesn't work. This is most likely wrong information, as shown in this very recent study [0].

[0] https://onlinelibrary.wiley.com/doi/pdf/10.1111/jsr.12712

Golem (decentralized computing for rendering, machine learning etc.) just went into beta yesterday.

You are right. Discovery, fraud protection and many other functions need to be there either way. When I say cost reduction, I mean getting rid of any additional premiums which eBay (or any corporation for that matter) levies because of their strong position in the market.

As for the problems of online poker, the Virtue Poker whitepaper is a good starting point about why blockchain is indeed that magic pixie dust[0]

[0]https://virtue.poker/wp-content/uploads/2018/03/Virtue-Poker...

A lot of valid points here, but ultimately very short sighted. Firstly, the author does not mention many of the highly plausible use cases for trustless and permissionless services such as replacing rent seeking P2P marketplaces like eBay (where the advantage is cost reduction) or online Poker sites (where the advantage is manipulation resistance). He does not mention DAOS, where strangers can trustlessly enter into contractual obligation to create e.g. a crowdsourced Venture fund. He does not mention the potential to provide financial services to the 2 billion unbanked people in the world. He does not see the potential for innovative and completely new financial products coming out because of the permissionless nature of public blockchains.

The use cases that he sites include buying an e-book using a smart contract or its use in the supply chain or voting. The e-book example is slightly misguided, because this is not an use case where decentralization provides much value. So the entire discussion about auditing smart contracts is rather unnecessary. But it is nonetheless an important objection that may become important in DAOs. But it's short sighted to think that this problem won't be solved. Some companies are already starting to tackle the smart contract to natural language translations (e.g. Iolite).

Supply chain is yet another use case where it is unclear whether blockchains will add value. Perhaps cost saving. But note that the blockchains used in supply chains will most likely be private, therefore permissioned. It's unfair to put private and public blockchains in the same basket, because they are very different beasts.

Finally the use case of voting. The author thinks that whoever is in power might distribute a few additional addresses to their cronies to manipulate the system. He is missing the point that all data is public in public blockchains. So anyone can audit the voting results later if they want and discover that those addresses do not belong to any real people (assuming there is a KYC type identification system built-in). Voting is an use case where blockchains are sorely required. All we need is for the knowledge to reach public consciousness and for the people to tell their governments that they want it.

Hats off to you Ines and Frederique. You guys do really great work.

I love how the SpaCy related websites are always so well designed. Their dependency graph visualizer is just amazing. I know that Ines is behind that one, but don't know about the other stuff.

Now coming back to the topic, I have so far just used Jupyter Notebooks and spreadsheets to do annotations and by golly, it is an extremely boring and tedious process. This looks like a fun tool to try out for my next NLP related project. Might spice things up!

But I hope that like all SpaCy related ideas, it doesn't assume too much about the problem at hand. I usually use NLTK instead of SpaCy because it allows me to be very flexible, except for the sentence tokenizer, where SpaCy's accuracy is hard to beat.

> Rarely on top but still a high ranking relatively speaking(2-4 etc.)

If you look at the beginner book section, the kid's section and the application walkthrough section, there are almost no Packt books there. In the reference book section, the Packt book ranks last among 6 listed books.

In the individual topical sections, there are on an average 5 books listed per topic, so 3-4 is a pretty low rank.

The only two exceptions are the ML book by Sebastian Raschka and the high performance book by Gabriele Lanaro which are ranked in the 2nd position. Both of these are outstanding books in their own domain and deserve their spot.

I see your point about Powerful Python having another retail outlet, but I think the same is true for all the other books because they too have other retail outlets.

If you take a look at the website more closely, you will find that Packt books are rarely on the top. Your comment is actually well reflected in the ranking.

You had a problem with the Dusty Phillips book ranking on top of Powerful Python. And I can personally guarantee that this book is out of the world good and deserves that spot, if not higher (though the book above by Luciano Ramalho is also very high quality). Here is my review of the Dusty Phillips book, in case you are interested [0].

On your last point, it is certainly best to judge a technical book solely by the content. The problem is : you don't have access to the content until you buy the book. You can of course take a look at the TOC, but doing that for all books in a given field would take a lot of time. It is always nice to have some other indicator that correlates well with quality e.g. average reviews or popularity so that you can reduce the problem space. This is what PythonBooks aims to provide.

[0] http://pythonbooks.org/python-3-object-oriented-programming-...

I love it, especially how you summarized the key points in one or two sentences instead of going into lengthy stories.

It seems the key strategies can be listed as follows:

1. Great video which demonstrates a good knowledge of the community (Dropbox)

2. Viral (aka sharing) features built in (Instagram)

3. Leveraging existing userbase (Instagram, Product Hunt)

4. Exclusivity (aka invite only) (Spotify, Facebook, Quora)

5. Reaching out to competitor's users and doing a better job than the competitor (AirBNB)

6. Getting the tougher side of the marketplace on board first (Tinder)

7. Offering rewards for signing up etc. (Paypal, Yelp)

8. Starting with a premium service first to deliver best user experience, leading to word of mouth (Uber)

9. Contests (Youtube)

10. Influencer outreach (Etsy)

11. Seeding site with good content (Quora)

Do continue with this site. I find it to be a very valuable resource.

Some very good points indeed. Let me address them one by one.

> Why should popularity automatically equate to quality?

Popularity is the by far the best indicator of quality that I could find. HN, for example, also operates on this principle. The posts with the highest votes get to the top. However, this does not mean that the posts that do not get as many votes are not as high quality. dang has mentioned this several times and went on so far as to say that high quality posts getting overlooked is one of the biggest problems of HN. One of the ways HN solves this problem is by using moderator discretion to put some overlooked posts in the front page, overriding the algorithm (I believe this post was one of them, it had just 5 points when it was put on the front page). In the same way, I use some editorial discretion to penalize books that have typos and poor English, if I could detect this by going through the Amazon preview. Some books were not even included in the list for this reason, most of them being independently published books. However, the book by Narasimha Karumanchi escaped my attention. Thanks for mentioning that it has typos etc. I will review it again.

However, just as HN tries to provide value by best effort quality ranking, PythonBooks also makes a best effort attempt at judging quality. It is not perfect, but it's the best I could do. If you have a better idea, let me know. I am all for improving the site.

> However Packt Publishing are generally not very good quality publications

Don't agree with this. Many Packt books are high quality. I have read several, including the one by Dusty Phillips, which is above Powerful Python. I do not share the same prejudice about this publication.

> Possibly because the author also sold the book DRM-free on their own website?

Possibly also because the book is relatively new and will only get more popular with time. The scores are dynamic and changes every week when I sync my database.

Like Powerful Python, most other books are also available outside Amazon, so it's mostly a level playing field. The exception is genuinely free books like Automate the Boring Stuff. These books get a blanket score bump to adjust for this.

Hope I could answer some of your questions and thanks for taking your time to provide feedback.

Yup, it is on Amazon now, and so are many other self published books.

You can see that. On the pages that list books, there is a Python version filter. In desktops and tablets, it appears on the sidebar to the left. On mobile devices, the filter is collapsed and appears on the top right. On the book pages, the Python version information is available on the sidebar. On mobile devices, click on the Vital Stats link on the top to see this information.

Thanks! I had a lot of fun making the site. I am also committed to keeping it comprehensive and updated. Made a powerful admin interface so that syncing would only take about 3-4 hours per week of my time. There's about 600 Python books out there according to my best estimate, so there's a lot of ground to cover.

That's actually a very good point because people are more likely to look for recent books. Will change that in the next update.

Thanks for pointing that out. That book came out very recently (a week ago?) and my database is a few weeks old.

Glad that you find it helpful.

Good point. I bump the ranks of the free books slightly to adjust for that. Similarly, for books that have multiple editions, I make some adjustments to avoid penalizing these books for fragmentation.

Two Scoops of Django is one of my favorite books! As to how popularity score is calculated: I used the Amazon Bestseller Rank of the book to assign a score for popularity. This rank loosely indicates how many copies of the book were bought recently. I found this to be well correlated with quality (I know many of these books by experience, and I found that the ones that I already knew to be good books rose to the top). In addition, when you filter for topics in the beginner section, another score is assigned based on relevance to those topics. This was a bit more work. I went through the contents of the book manually and tagged each book by the topics it covers and the percentage of the book devoted to that topic. After the relevance score is computed, it is combined with the popularity score to get a final score.