One of Tolkien's goals was to create something which resembled history rather than literature. Or if not history, then mythology that plausibly belongs to a real place, rather than a neatly-constructed story. So, some of the things that he does are "janky" from a purely dramatic perspective, but fulfill that purpose.
A common example from Lord of the Rings is having many related characters with the same or similar name. The convention in literature is to make characters distinguishable for practical reasons, but in "real" history naming conventions are messy and people aren't always distinguishable. There are even real debates among historians about whether a name refers to multiple people or not!
Tom Bombadil is a similar thing. He serves no dramatic purpose. But a history, or a mythology, doesn't always have events that fit together in a neat way. Sometimes stuff just happens, or there are things that are only tangentially related. And it's difficult to draw a circle around events and say inside the circle is our story, and outside the circle not our story. There are other stories, intersecting.