In Tokyo, it is passably fine. Business communication is strongly preferred in Japanese. However, learning resources are many & people are generally helpful and accommodative. 20-30% will be proficient in English. I am told Osaka-Kyoto work setting is more traditional than Tokyo, so expect less foreign language fluency.
I wouldn't discount the difficulties of not knowing the local language. Living in Japan is not easy if you do not speak or understand Japanese. Every single form or application is in Japanese. All government related procedures are in Japanese. Banner signboards & instructions too. You'd miss out on several important bulletins e.g. tax instructions, license updation etc., if you read only the EN versions, as Japanese ones are more detailed & explanatory. One-to-one translation of information is not guaranteed with such documents.