You are ignoring an important factor: If you don't have enough calories the body will switch to starvation mode, you'll have low energy, be cold, etc.
The net result is that, unless you are actually starving (which is bad for you), you'll actually lose less weight that way!