My direct experience of working with programmers over the years is that this will generally descend rapidly into an inefficient mess. Yes – maybe it works, sometimes. I'd argue that's the exception, rather than the rule.
If you work with people who dont care, youre screwed. Nothing will ever work to force them to actively engage with the process of creating good software.
..but, surprisingly, when you do work with people who do care, they self organize remarkably well. The Valve megacorporation is fundamentally based on this. It works. Thats been my personal experience, over the years.
The problem that has to be addressed is convincing people to care, and self improve.
Forcing a process on the unwilling is going to fail no matter what you do.
So where's my Half Life 3, dammit ;-)
I don't agree that people who do care necessarily self-organise well, or that this self-organisation is compatible with other business goals.
Sometimes it works, and a group of engineers et al. do a good job of forming a coherent team with minimal process. Valve's maybe an example – I don't know. My experience is that this is the exception, rather than the common case.
A good process can provide tools that help good people who care communicate and deliver more effectively. They help us to keep track of how we're performing and analyse pain points, and to make sure that there are methods for dealing with common problems.
For example, if my team felt that daily stand-ups were an inconvenience that weren't helpful, then we wouldn't do them. That's the goal; there's no cast-iron set of rules, just a generally applicable set of principles to follow.
Some people are self motivated and other needs their slap in the morning (for some, that's a couple of times daily).
Why is Valve a megacorporation vs. just a corporation?
Experience needed (read: got fucked before, now can avoid the treacherous waters and apply effort where and when meaningful).
Ability to deal with herding cats much needed as well.
Add relentless ability to move forward and not hold grudges.
Emphasis mine. You probably have more experience than I do, but largely I've seen it to be true that skilled (or, in the absence of skill, passionate) programmers tend to self-organize, as long as they all have the same goal (which hopefully is true for the majority of your team - if not, no methodology is going to help you).