The Toyota Way at Codeweavers
infoq.com
infoq.com
The thing that businesses don't want to hear is that software development is a skill game. You win by having better players. You cannot put blind process around every facet of it. I know you'd like to, so that you can then pay very little for unskilled labor. But the fact remains that you cannot, even today.
Wysiwig html editors and drag and drop UI builders for rapid application development were trending for a while in the 90's. Then everyone realized that giving these tools to monkeys resulted in a big pile of poo. So the trend reversed back to hand coding UI layouts using markup, separating business logic from display concerns, etc
I guess what I am trying to say is that if there is any hope in putting a lot of process rigor around software development, it will be enabled by tooling. Until then, skill and experience matter. And they aren't cheap.
Put another way, if the Cleveland Browns stole the New England Patriots playbook, they would still suck.
I also think that languages like Java have become popular in big enterprises filled with lots of coders because the language makes it hard to write difficult to maintain code by doing away with any features that would make it easy to obfuscate the intentions of code. This concern also had a big influence on the design of Go and all the features it left out.
The upshot of these coding methodologies that stood the test of time is that they protect the project from bad coders and they prevent really smart people from writing code that is too clever such that noone else can maintain it.
Effectively, these methodologies are a kind of bureaucracy. Besides self- propagation, the primary purpose of bureaucracy is to cover for incompetence. So testing and version control are (in part) a way to cover for incompetent coders.
Agile philosophy is loosely similar in that it has a “do what works without dogma” core, but has much weaker focus on actual concrete measurements and empirical validation, which is one reason why despite its fundamental (in the Manifesto) opposition to canned, one-size-fits-all process that isn't adapted to the particular context, it devolves into a lot of that—without a cultural value of empirical measurement of outcomes, social proof and “this is the way to do it because this is what other people who succeeded were using”, or “this is the way to do it because it comes from <authority figure> who makes <abstract argument> and/or gave <subjectively appealing presentation>” become the bases available to make decisions on how to implement the vague ideals of the Manifesto. And also why it can be particularly hard to sell to an organization that does believe in changing process based on a specific, testable hypothesis about improvement in a key measure.
I found this really interesting. I completely agree that the principles behind the Toyota Production System can be applied to many other disciplines than just car manufacturing. I wonder if Toyota itself actually applies its own principles when it comes to software development. When their software got audited as part of the unintended acceleration case [1] it was described as spaghetti code and having generally poor engineering practices.
I'd be interested to know why this seeming discrepancy exists between Toyota's quality in hardware vs software.
[1] http://www.safetyresearch.net/blog/articles/toyota-unintende...
The software you would use to configure the parameters in the car (e.g sensitivity of headlights turning on, whether or not it'll chime when you're not wearing your seatbelt) hasn't even been ported to 64 bits yet. It seems to have been written by one of their O.E.Ms (Denso) and it seems like the cars have used many of the same design patterns since the 90s.
Look no further than the infotainment system, it's always a generation behind other automakers and we all know that most automakers don't set the high bar of consumer technology.
Software however is all but impossible to spec from the beginning, or perhaps ever at all... so with only working with the smaller details, the larger picture is lost. At any time, any team, is instructed to stop a problem and immediately fix it, it's band-aids on top of band-aids.
Having multiple teams working on overlapping projects, with little communication leads to the spaghetti code that was mentioned.
Much of the code being dealt with by Toyota (at least that subject to the lawsuit and so made available for analysis) was embedded code which is a bit more amenable to very deliberate design, testing, and verification. NASA, for instance, has a long history of producing extremely reliable software, even proving the software correct at times. Would any private corporation in the world ever resort to following such a practice? Never. It would quadruple their software spending and, maybe worse, they would have to listen to their engineers. When the business wants to ship a project, if the engineer says 'its not ready yet', if its a bridge and they go ahead someone in a suit goes to prison. If it involves software, everyone goes home happy except the dead customers.
Embedded software can be spec most of the time and it should be.
1: https://www.engadget.com/2017/05/31/toyota-automotive-grade-...
One of the largest historical sources was inspection and repair: you built something and then inspected it and fixed any problems. The inspectors and repair people didn't really talk to the production people much.
In Lean Manufacturing, the main purpose of inspection is to drive process improvements back into production. (This is the simplistic explanation, but I'm only using this as an analogy)
Thinking about data management - there's a lot of dirty data out there, and a lot of data munging to fix it up. Lean Data Management would drive process improvements into data acquisition.
Of course on the other hand, it's extremely cheap to fix data, on a per-unit of data basis. It's rule management that's expensive.
Toyotas system is predicated on the idea that for manufacturing THEIR product the touch time is way less the than the buffer/quueue times ( ie, the time spent adding value is small compared to the the overall time to get an order and get it delivered. Eliminating waste is a means to an end to cut into unecessary buffers and queues and increase flow based on the constraints of the system.
In software there's a lot of applicable ideas, but it's not exactly the same and requires careful thinking around what your revenue model is and how you deliver software based on that. But the focusing idea is not the practices of lean but how to increase flow and build (reuse) practices around that.
There are benefits for examining the flow of deliverables and information, and trying to eliminate waste, but much of the TPS is geared around the manufacture of physical objects and it is a stretch to apply it elsewhere.
This Toyota methodology is about taking things that have already been designed and then incrementally improving them or figuring out better ways to put them together.
Building software is more akin to the process of taking a blank sheet of paper, a knowledge of what kinds of parts already exist, then designing a new car to fit a new market segment.
What might be more suited is operations development. It's quite similar to DevOps, as is, with DevOps just having been adapted more explicitly to software.
For the actual design of individual pieces of software, I wonder if something like Toyota Product Development System is a better fit.
While it helped our customer service department quite a bit as-is. The Toyota Way is definitely not something you can directly copy/paste and use in software development. Building a Camry 5000 times is not the same as shipping 5000 different software projects, so a lot of the analogies and examples you find in text don't always apply. I always found it amusing that you never see or hear examples used from the R&D and engineering of the actual automobiles -- just once all that hard stuff has been figured out and the manufacturing starts, all the principles are magically, easily applied.
To be honest, a lot of it is just "common sense" or things that just naturally transpire if you have a good team (especially if Agile dev and ideas like continuous delivery are used anyway.)
With the negatives aside, the problem-solving culture part of it is something that can really change things for the better and is worth researching. If nothing else, I found just being able to have a framework for solving problems as they come up is huge -- surely regardless of what a company does. For instance, integrating a standard to collectively write out the "Five Why's" and clearly define the problem statement when a team is gathered to discuss solutions to a problem can save so much time. How many times have you been dragged into a meeting, go through a huge discussion, design a solution, only to realize later that it wasn't really solving what needed to be solved?
Outside the culture aspects, it mostly just felt forced, frustrating devs, and really made things feel almost like a cargo cult waiting for the goodies to drop from the sky.
I'll pass, thanks. I know it's all fine for Toyota because the courts ruled essentially "Its computers and nobody knows how they work! We can't convict them of criminal negligence!" and it saves them billions by not hiring experienced people or giving engineers the tools they need or training them or anything like that... but I actually have a conscience and don't mind if I make a little bit less money if it means there won't be the dead bodies that are my responsibility like Toyotas got.
https://www.amazon.com/Lean-Software-Strategies-Techniques-D...
The West has a tendency to cherry pick techniques from holistic approaches (for example quick path to mindfulness vs meditation and Buddhism). I suppose it's a cultural difference. In my opinion, it is really worth it to try to understand Lean as a whole instead of looking at kanban only.
I agree that it's worth it to spend some time to understand the holistic approach... but there's also a lot of bullshit and over-reliance on form over function.
The “bullshit” emphasis on form before function is a pretty standard part of a 3 stage learning method - naive and unquestioning memorization of rules/form, unique combinations of the memorized forms, and finally throwing it all out the window and inventing new forms with the newly acquired knowledge and skills.
Honestly, over the decades, there are very specific patterns in all the ism's, and its the recognition of these patterns which each new generation of ignoramus decides to make their own and turn into the Hot New Cool Shit.
So I feel its really necessary to point out that while, with the Toyota Method, there is very definitely a refinement of techniques that have been well-worn, and proven, indeed in many markets around the world - there are still parts of the human psyche of productivity which remain unexplored.
One of those aspects, I feel, is the obfuscation and neglect which occurs with every wave of computer culture.
We kind of don't really work on the reasons for obfuscation - because obviously this is not productive. Productivity means, getting the shit done, in a way that increases value - for the individual producers as well as the owners of the means of production.
Toyota works. The proof is in the fact that it is one of the biggest and best brands in the world.
But what doesn't work is the occlusion of methodology that occurs, which makes the revelation here in the 21st Century of time-worn, centuries-old methodologies, relevant.
We get this way because we decide to occlude, forget, neglect, and ultimately .. reject .. the ways of the masters. How many of us read this article, gave it a cursory glance, and decided to close the tab, for whatever reason: "its too preachy", "its a culture thing", etc. And what number of us actually decided to incorporate the Ohnno way into our thinking, for the simple fact that HN gave us the clue?
Really, what I would like people to understand is that there is a desultory, occluding factor, to these different 'ways'. The moment the marketing people take over a working technology, they decide to brand-differentiate, and this means forgetting the roots of it all. "Nobody thinks of the sugar; all we see is the cola" can only happen if we actually act on the basis of ignorance.
The point being, the Toyota way seems - like so many other ways - to have the principle purpose of fighting ignorance. Ignorance of oneself, ignorance of others, of the customer/user, of the managers, and so on. Ultimately, the Zen approach is, remember that you will always be forgetting, and express your current reality in such a way to defeat that. No successful brand is ever forgotten; no successful methodology can happen in a vacuum. Yet: it requires such a vacuum before you can find your own handle on it, and ultimately apply it with success.
"Applying it with failure", on the other hand, is a sure-fire to re-invent something else, 'brand new', again and again.
Don't you feel there is something inherently intrinsic to Toyota Method? Well, there should be a list of these feelings, by now, for those of us who have been paying attention for decades ...
The paradox is that the more you attempt to tie practical/tactical things like Lean, Kanban, meditation, diet, excercise, etc to a broader perspective, the more likely the average person will actively resist/ignore both.
though I'd be remiss if I didn't point out that team jack hollis is on a path to do that yearly. if they follow through on the user-based stuff, they'll end up doing exactly what you were hoping for, @mmjaa.