For fun and pedagogy, I did build some speakers recently and it was educational. I bought full-spectrum speakers (IE, a speaker that handles both highs and lows, rather than using tweets and woofers and a crossover) and a simple stereo amplifier breakout board from Adafruit. 3D printed some speakers, and did all the soldering/wiring/enclosing myself.
What did I learn? That building good speakers is hard! There definitely are parts of the spectrum the speakers don't handle well (low frequencies; I can't hear above 15khz any more). But what I really noticed is that because I didn't do any isolation (no padding inside the speaker enclosure, the speakers are directly on my wooden desk) I can hear a lot of artifacts. I'm actually glad I can notice them, because in some sense it validates my own sense of hearing, as well as my understanding on how to improve sound.
If I were to do it again, I'd add a crossover, subwoofer, tweeters, and a ton of physical isolation (replace the 3d printed speakers with wood, add padding, and a mount under the speakers to reduce the table vibration.
One of the best speaker systems ever was an absurdly simple system: the Grateful Dead's Wall of Sound. Because large auditoriums with PAs had terrible sound, they did something nobody else tried: a huge array of speakers, each band member with their own subset of speakers (no mixing). It was absurdly expensive but it had a very high quality sound field even at low volumes (very clear 1/4 mile away).