> Tensor product is basically the mathematical way to express if condition.

Not really, no. The way mathematicians actually express if conditions is with the word "if". The obvious pointlessly formal way to do it is with a pair of functions `ThingConditionedOn -> {0, 1}` and `{0, 1} -> Result`, but why would you?

> dealing with tensor matrix is simply a waste of memory since 90% of the time people are dealing with sparse system.

Tensors are not their components, any more than locations are their coordinates. Whether you choose a sparse or dense (or symbolic) encoding does not change the object being encoded.