I think an accent is perfectly fine (everybody has some kind of accent), as long as it doesn't significantly increase the effort for others to understand you. From my experience listening to some French people speaking English, you actually have to remind yourself that they are speaking English and not French. They might use English words and grammar, but it sounds like French, flowing from sentence to sentence with the typical melody.