You could answer their English in Norwegian, or even some other language, e.g. French. If you have started the conversation in Norwegian, they should not assume you can speak English anyway.
When I go to Spain, I will always try to speak Spanish and if they reply in English (which they often do because I make mistakes and/or have an accent), I just keep replying in Spanish. That may be seen as "rude" or inefficient by some (although others probably appreciate the effort), but it's really the only way you can keep making progress. (Of course, if it really gets too complicated you can still switch - but usually people switch way before they've reached the end of their skills.)