Most of the services just ran normally on a developers system, or we had a docker script that could spin things up locally, or you used one of the development sandboxes running on the system.
BTW, we had a system that allowed very low cost creation and management of containers (VMs in our case). I think that is an important detail.
Could you use a debugger for a feature? Most likely not, because a lot of our features already pulled data from external sources which you had no visibility into.
> You're talking like there was a single "owner" for each service? What did you do when they went on holiday / left? How did people share knowledge about the system?
Same problem every development team has. Who setup up the CI server? Who setup the front-end proxy? Who configured the AWS setup? Everything needs redundancy. We had a list of all tools and services and had multiple names assigned to each one. Even without services a team should have this.
> Dealing with an outside service brings a lot of overhead. It's harder to debug, harder to test. When things are maintained by a separate team it makes sense to treat them as an outside service, but within the same team why would you make all that extra work for yourself?
Because it worked really well and we didn't see that extra work. I disagree that external services are hard to interface to. They are hard to work with when they are buggy, but if they hit the specs, they generally are not an issue. I have no problem interfacing to S3. Because of the loose coupling, things were very extensible and easy to modify. A few times we would replace a service with a new one by adding the new one and slowly migrating everything over to use it. In some cases, that migration took over a year, but that was fine.
> Sounds like you need a better language. I agree that anything that might segfault belongs in a separate service, but most of the time the solution is to not use stuff that might segfault.
LOL, it was Python. Bugs happen, things break. Everything can eventually segfault for all kinds of reasons. Run a complex system for a decade and you will see all types of issues. I remember running OpenVZ before switching to KVM and occasionally we would even see kernel panics.
> How does that fit with "services only had to talk to other services from the same commit"?
You conveniently left out "or branch". Listen, I am not trying to sell anything, I don't care what you think. I am just sharing my teams experience. It was overwhelmingly positive using services to build a complex system with real revenue from Fortune 100 customers for almost a decade.