Won't it be a problem when looking for funds though? People who can fund startups tend to be a bit older and might not have a good level of English (or will simply be more likely to help the startup if they can communicate in French).
The real challenge for a non-French speaking founder would mainly be in dealings with local authorities. You can't expect public servants to speak any English or have any English-translated documentation / forms available.