I successfully removed it years ago because Typescript replaced everything I was using Babel for, things even build way faster and I even got Intellisense autocompletion and refactoring as bonus.
You can have the best of both worlds: Intellisense based on types in the editor yet fast transpile in the build.