>Does the program halt when the power cuts out?

Yes. A computer can be in an infinite loop, lose power, and then stop functioning.

>how do you define infinite loop and forever

Infinite Loop: A system is in an infinite loop when that system keeps returning to the same state making it unable to progress (depending on the context the definition might change to where making progress is okay and it in only returning to a partial match of an existing state (think of an echo server for example))

Forever: A time period that ranges from t=now to t=infinity.