Hacker News Reader

For the Information Gain calculations, why are the chocolates we don't want to eat (Blue & Kit Kats) discarded, i.e. considered entropy 0. Shouldn't it be included as part of the Information Gain formula?

View on HN

Since we are looking at entropy with respect to class (whether we want to eat or not eat), entropy for blue & kit kats is 0 as we don't want to eat them at all. So there's no randomness.

They are included in the IG formula but since they are multiplied by 0, they don't have any effect.

Reply on news.ycombinator.com