And that's how it should work imo. But if you enable esm (which you might need in the future because of packages being esm only) you can't use those, only .js.
That's because typescript developers are dead set that they don't want to transpile the imports, they just want to copy paste them into the resulting file when running tsc.