It's true that Japanese tend to be more strict about accurate production of phonology than many other language speakers but speaking and writing are huge enablers of becoming better. It's really not that hard to unlearn bad pronunciation especially in an immersive context. Also most Japanese have a tendency to gently correct a speaker if you use the wrong phrase, particle, or construction.
Obviously if you've been self-learning your first few conversations with real people are going to be rough and so maybe avoid dense topics like Japanese attitudes on the JSDF. But if you end up in a light conversation circle where you talk about your favorite foods you'll be fine.