You seriously underestimate this in general case. Build system may be made in weeks, but is polished in months or even years, to account for all the different usage and environment scenarios. Otherwise, it's typically very fragile.
After having been burned several times by images I prefer writing my docker images from scratch (based on the Debian or Alpine images) for production systems. I only use ready-made images for quickly getting something running locally to evaluate it.