Nope, lots of casual sports teams will have people say, "hey we couldn't really socialize during the game, let's go grab a beer afterwards?". Hiking is similar, "hey let's take a break and eat some lunch."
If you do order out, get a salad and skip the dressing.