You would be shocked how many calories restaurant food has (usually >1k calories in a single dish -- no matter what kind of restaurant -- and probably more in the US). Personally, I would've assumed this is the more likely primary factor for your weight loss (combined with the fact you probably eat less food overall because you only have two meals when doing IF). You could try going to restaurants again for a few weeks and see if you start gaining weight again.
But, if it works for you -- it works for you.