I share the concern about mono-cultures in tech, but if you are serious about getting into data science or AI I have to agree with the OP -- stick to TensorFlow. TensorFlow.js is not a first class citizen of that ecosystem, but its close enough to get real work done.
Also... there are some groups that use Javascript for this kind of work (We use Typescript for our projects), but the vast majority of it is in Python. As much as I dislike that language, it completely dominates AI and data science.