Hacker News Reader

Hacker News

Comment by darkxanthos | Hacker News Reader

Full thread

darkxanthos

Does it seem weird to anyone else that the first example is approximating XOR?

View on HN

No, that's a pretty standard trivial example for NNs.

http://en.wikipedia.org/wiki/Feedforward_neural_network#Mult...

The reason why it's used is because it's a very simple example of a non-linearly separable function.

You can interpret boolean functions as classification problems: given some input(s) x, classify the output as either 1 (class A) or 0 (class B).

The truth table of an XOR function looks like this:

  x1 | x2 | XOR | Class
  ---------------------
  0  | 0  | 0   | B
  0  | 1  | 1   | A
  1  | 0  | 1   | A
  1  | 1  | 0   | B

If you were to plot the XOR function in R2, it would look something like this:

  x2 ^
     |
     |
     A     B
     |
     |
     B-----A----->
                  x1

To be non-linearly separable means that there is no straight line that you can draw in the above plot that will split the classes. Thus, we need a non-linear classifier, e.g. a neural net.

Would you mind elaborating the linear separable part?

As in, the following would be linear separable (all labels on the same side of the line)?

   x2 ^
      |
      |
      A     A
      |
      |
      B-----B----->
                   x1

It's a bit hard to elaborate. You have to rember that a classification problem may be generalized to more than just two dimensions, in fact it can be generalized to however many dimensions are necessary to describe the data. In the general multidimensional cases linearly separable relates to the number and type of hyperplanes (subspaces) necessary to correctly classify the data.

In this simple 2-d case the subspace is a line, so this example is linearly separable. In the 3rd dimensional case linearly separable refers to classification using a single 2-dimensional plane. For 4 dimensions with the data, you need a single 3-dimensional hyperplane for it to be linearly separable, and so on.

generally a linear function takes the form of:

v(x1,x2) = C1 * x1 + C2 * x2 + C3.

C's being "coefficients".

In your case, v(x1,x2) = 0x1 + x2 + 0

i.e. C1 = 0; C2 = 1; C3 = 0

suffices, where a result v > 0.5 indicates A and v <= 0.5 indicates B.

Interesting. Thanks for that

Reply on news.ycombinator.com