Well, that just means that an regular (x/k)ubuntu isn't necessarily the distro you're looking for. Back in the days when I was using Linux on my main computer, canonical shipped a stripped down version of ubuntu (ubuntu server maybe ?) where you could install just what you need and nothing else (I beleive it came without a window manager for instance).
What you're describing was pretty much my Archlinux setup btw. It was just more convinient for me to build it from an archlinux than a regular ubuntu. You could with an ubuntu though, it'd just be a lot more work I feel like.
EDIT: turns out I'm pretty much explaining you what you already know.... I'll leave my comment up anyway, maybe it'll help someone else.
> My personal opinion is that i3 rocks, and and that I don't need all of Ubuntu or Xubuntu