Check out The Complete Guide to Intermittent Fasting. The research he cites shows that the effect you describe occurs in caloric restriction diets, not fasting. Fasting increases HGH and maintains BMR.
Perhaps what you are describing is the early days of fasting when your body is becoming fat adapted and you feel a bit sluggish, but after a short period your energy levels return to normal (for me it was less than a few weeks, 5 years ago, and my wife just started fasting with me and she is adapted and back to full energy while fasting after two weeks).