That sounds lovely - but I think the legality thing is something to investigate early. I think a lot of places have rules about restaurants having hygiene certificates and passing inspections - maybe having a meal exchange would bypass that (like you can try other peoples' food as long as you also host every so often?). If you're taking money, I'd talk to a lawyer early (or friendly person with legal knowledge at least!)

Also, it might be worth starting on a specific scene first so you concentrate your community (your local town, your uni alumni, conference attendees at local tech conference or whatever).

edit: OK actually people do this, trended in London a year or so ago (sorry for non-reputable source: http://www.dailymail.co.uk/femail/food/article-1253584/I-ope...)