To Improve Quality Don't Measure Quality: Misunderstanding Quality (Part 11)

In this episode of Misunderstanding Quality, host Andrew Stotz and Bill Bellows discuss what not to measure when it comes to quality. Bill offers some great examples to show how organizations get it wrong, and how to get it right.

TRANSCRIPT

0:00:02.4 Andrew Stotz: My name is Andrew Stotz, and I'll be your host as we dive deeper into the teachings of Dr. W. Edwards Deming. Today I'm continuing my discussion with Bill Bellows, we're gonna have a lot of fun, who has spent 31 plus years now that it's 2025, helping people apply Dr. Deming's ideas to become aware of how their thinking is holding them back from their biggest opportunities in the episode, today is episode 11, and the title is "To Improve Quality, Don't Measure Quality". Bill, take it away.

0:00:35.6 Bill Bellows: Thank you, Andrew. And, so the title of episode 10, came from chapter 10... Chapter 1 of The New Economics, and I used a quote from Dr. Deming, which was, "Are you in favor of the improvement of quality?" Which Dr. Deming says, "Are you in favor of the improvement of quality? We can have a national referendum, yes or no?" Everyone says yes. Then he says... Then he say, "We could have a secret ballot." And... But I... At the beginning of the podcast, I had said, "Are you in favor of quality?" And it's... No, it's, "Are you in favor of the improvement of quality?" And so today I wanna, in episode 11, share it with our listeners and viewers, more of the profound insights from Genichi Taguchi. But I think, what I was just thinking is saying, "Are you in favor of quality?" And I've used that quote, which now I now realize it's a misquote. It's not, "Are you in favor of quality?" It's "Are you in favor of improvement of quality?" But in seminars, what I've done is used the quote, the misquote, I would say Dr. Deming would ask, "Are you in favor of quality?" And he would say, "We're gonna have a secret ballot. Is everyone in favor of ballot?" In quality, everyone says yes. So I would go through that.

0:02:16.3 Bill Bellows: And then I would go to the next question, and I would say to the audience, I'd say, "Okay. Dr. Deming made reference to secret ballot. So I wanna do a secret ballot. I want you to close your eyes, and I'm gonna ask you a question, and if your answer is yes, raise your hand. But I want you to close your eyes when you raise your hand, 'cause I don't want you to raise your hand 'cause everybody else does. Okay, so close your eyes." And I say, "Are you in favor of teamwork?" And all the hands go up. [laughter] And it's not so much "Are you in favor of improvement of teamwork?" But it's the idea that, acceptability saying this part is acceptable, as we've shared in prior episodes, is the essence of looking at that part, my task, my effort in isolation. And what that has to do with teamwork, I question. Now, with a few of us at Rocketdyne years ago used to talk about, we would say, you give out a term paper assignment, the term paper must be between 10 and 20 pages long. And what happens? They're close to 10 pages. Then I would share, we'd tell Allison, our daughter, I'd say when she was in high school, "Be home by between 8:00 and 10 o'clock," and she shows up around 10 o'clock.

0:03:51.6 Bill Bellows: And I would show a distribution over there. Then I would say, "What about a machinist? The machinist is given a hole to machine. And what does machinist do is machine the hole on the low side, and then a machinist is machining the outer diameter of a shaft or a tube. And what does machinist do? Machines to the high side." And so I would show those four distributions either on the low side or the high side, and say, "What do they all have in common?" And people would say, "Each of those people's looking out for themself. They're focusing on their work in isolation." Then I would say, "So what do you call that in a non-Deming company or in a... " In the first podcast there is a, called it a Red Pen Company or a ME organization, or a Last Straw companies... What do you call that behavior where people look at the requirements and say, "What's best for me?" What do you call that? What do you call, people scratch their head? We say... You ready? "Teamwork."

[laughter]

0:05:00.6 Bill Bellows: And everybody laughs. And then I turn to somebody in class and I say, "So Andrew, are you a team player?" And Andrew says, "Yes." And I say, "Andrew, if you machine the holes to the low side, are you a team player?" And you might say, "I'm not sure." And I would say, "Say yes." And you'd say, "Okay. I say yes." And I say, "Okay, Andrew, who's on your team?" And you say, "Me." "So, oh, you are a team player, man."

0:05:24.2 Andrew Stotz: I'm a team player. Team Andrew always wins.

0:05:28.2 Bill Bellows: Yeah. And I would say, so I say, "In a non-Deming company, everyone's a team player. All right. But who's on the team?" So I would say to people, "You'd be a fool not to be on your own team. The only question is, who else is on your team?" All right. Back to Dr. Taguchi to improve quality, don't measure quality. And I was, got into this in an explanation with some others recently, and somebody was showing me a bunch of defect rate data involving some process. And the question was, how to apply this occurrence of defect rate data to Dr. Taguchi's loss function. And so, again, reminder to our listeners, acceptability is everything that meets requirements is okay. Either I am unaware of differences or the differences don't matter, any parking spot, any professor any Thermo 2, any doctor and desirability is "I want this doctor, this parking spot, this, this, this, this, this." And so not just anything that meets requirements.

0:06:50.3 Bill Bellows: And Dr. Taguchi's work has a lot to do with that thinking. And Andrew, yeah, I'm on a month, on a regular basis, meeting more and more people that are listening to the podcast and reaching out to me on LinkedIn. And one shared with me recently then, and he started to listen to this series, and he said, he never thought about desirability. He says everything he knows, everything he sees every day, is acceptability. And he's like, "You mean, there's more than that?" And it's like, "Hello. That's what our series is trying to do." So...

0:07:26.6 Andrew Stotz: And let me introduce you to door number three, which opens you up into this whole 'nother world of...

0:07:35.6 Bill Bellows: Yes.

0:07:35.7 Andrew Stotz: The interconnectedness and understanding quality from the impact on all the different parts of the organization, not just the one thing and the one area. Yep.

0:07:46.6 Bill Bellows: Yes. Yeah, yeah, yeah. Exactly.

0:07:48.9 Andrew Stotz: But that's door number three. Now, we don't wanna go through that right off the bat, but when you go through it, unfortunately door number three disappears as you walk through it, and it's a wall...

[laughter]

0:08:00.4 Andrew Stotz: And you can't go back because now you understand that what is a system, what is the interconnectedness of everything, and once you see that, you can't unsee it.

0:08:09.6 Bill Bellows: That's right. Now, it's like, it's a holistic view in which... And a from a holistic perspective, parts don't exist, parts of exist, but everything is connected.

0:08:27.4 Andrew Stotz: Right.

0:08:28.2 Bill Bellows: And what does that mean? So anyway...

0:08:30.1 Andrew Stotz: And just to put that into context, let's just take a car. A customer never buys a part. And they don't buy a jumble of parts, they buy the car. So to the customer's perspective, it's even more meaningless, the independent parts of that.

0:08:50.3 Bill Bellows: When I would go to Seattle and do training when Rocketdyne was owned by Boeing, and I'd be doing training for people working on commercial airplanes or 737s, 47s and whatnot. And one of the jokes I would use is that, "Hey, 747... " People went, "What's a 747?" How about 787? If I was today, I'd say "a 787 is not a bunch of parts that fly in close formation." But that is, the mindset is that... But anyway, so acceptability is looking at the parts in isolation, looking at things in isolation, it's assigning a grade to a student, it's performance appraisals, that's all about isolation, it's thinking, "I won the game, I get an award. I lost the game." All of that thinking, from engineering to, how we look at human resources, the idea that the savings add additional only works when the activities are independent. So that's all acceptability, looking at things in isolation. Desirability in this idea of a preferred value, I don't know that anyone contributed to that, besides Dr. Taguchi. In fact, this morning, I was talking with some friends overseas about Joseph Juran's work. And, do you remember last time you and I worked, I was sharing with them that our last podcast followed the last meeting I had with these friends in Europe. And I said that conversation led to our podcast conversation about Quality 4.0, and it's all acceptability, acceptability, acceptability, meet, meet, meet requirements.

0:10:35.6 Bill Bellows: This very conversation. And I said, I went back and did some research on what Joseph Juran... How Juran defined quality. 'Cause I looked at the ASQs definition of quality and it gave two definitions of quality, one attributed to Juran talking about quality as fitness for use, and then Philip Crosby's definition is, meeting requirements. But you may recall, I said, there is no explanation of how Dr. Deming defined quality. Yeah, maybe that will come. But, so I was sharing that with them, and also shared with them a model I've used. And it might have come up in our first series, but I think the classic model within organizations is, I work, I follow a bunch of steps to make a part, a thing, a module, something. And if all the requirements are met, I hand off to you, you're downstream. And then likewise, there's others in parallel with me that hand off good parts, good things to you. Because they're good, we can hand off to you. And then the model is you take the parts that are good and put them together, and because they are good, they fit. And then you pass that integrated component downstream where other integrated components come together. And we progressively go from, it could be that we're putting together the fuselage, somebody else is putting together the wings, and it's all coming together. And at the other end, it's an airplane.

0:12:22.5 Bill Bellows: And on every handoff we hand off what is, so the parts that are good fit, the components that are good fit together with other, then we turn the whole thing on, it works. And I show this flow to people and I say, "So what do you see going on in there?" And what eventually they start to see is that all the thinking is black and white, because they're good, they fit, because they fit, they fit, and when you turn it on, it works. There's nothing relative about that. And so I was sharing that with these folks this morning, and I said, after you and I spoke last time, went back and looked, and Juran talks about fitness for use, and the question was, is Juran's definition of fitness, absolute fitness or relative fitness? Meaning that there's a degree of good in the parts associated with desirability thinking, and if we've got degrees of good in the parts, then there's degrees of fit. And, well, it turns out there's plenty of reason to believe that Juran had a model of acceptability that the parts are good, then they fit. All to come back to what Dr. Taguchi is talking about in terms of improving quality, is improving quality from a variable perspective that there's degrees of good. And so now we go back to, to improve quality, don't measure quality. And I remember when he said that and we were dumbfounded, "Well, what do you mean by that?"

0:13:52.5 Bill Bellows: And then he would go on to explain, that traditionally, we look at the quality... The lack of quality of something. An inspector says, "There's a scratch on the door. There's a ding here. There's a crack there. There's a, the weld has a drop in it. The weld has porosity." You know what that means is that's not a... The quality inspector is looking for the absence of a crack, the absence of porosity and things like that. And it also parallels with what I learned from Ackoff, Russ would say, [chuckle] "Getting less of what you want doesn't get you what you want." So you could say, "I want less waste, less defects." Well, what is it you want? Again, the clarification is, Russ would say, "Getting less of what you don't want doesn't get you what you want." And likewise, Dr. Taguchi talked about, what is the function of the process? So if you're talking about, imagine on a washing machine, when you have a... Or a dryer, and you have a motor that's spinning, and around the motor is a belt that's spinning the drum. Well, the quality problem, classic quality problem could be that the belt slips, or the belt cracks, or the belt is vibrating.

0:15:28.3 Bill Bellows: Well, then you say, "Well, okay, what's the function of the belt?" Well, it's not about cracking. The function of the belt is to transmit energy from the motor to the drum. And if it does that really well over sustained periods of time, then that suggests there's probably less cracking going on and less slipping going on. But if you don't look at it from a function perspective and ask, "What's the function of the belt?" And move away from, "Well, I don't want it to crack and I don't want it to slip." Well, then tell me what you want it to do. What is it you want it to do? Now, let's get into more of what we do want. And then, and this is what's neat listening to Dr. Taguchi as an engineer, you say, "Well, okay, so what is the belt trying to do? It's trying to transmit energy." So if I can design the belt, and by changing the materials of the belt to transmit energy, under wide-ranging temperatures, wide-ranging usage conditions, if I do a good job of that, then I should see less cracking problems. Absent that, if I try to reduce the number of cracks, I may end up with a belt slipping more often. So then what happens is you end up trading one problem for another, which is not uncommon.

0:16:57.7 Bill Bellows: You go from, the cookies being undercooked to overcooked as opposed to saying, "What's the role of the baking process?" And he would say, "To transmit energy to the cookie in the precise amount. And if we have the precise amount and distribution, then that should work out." Now, relative to welding. Welding, there is, there may be a dozen different weld anomalies that inspectors are looking for, with X-rays, they see porosity, they see, what's called drop-through with the material and the weld, drops a little bit, which could result in a fatigue problem leading to cracking. Well, here Dr. Taguchi would say, "Well, what's the function of a weld?" Say, well, to join two pieces of material together with a given strength. And so you join them together. And then once they're joined together, now you run tests and you say, "I wanna... " It could be, "I wanna heat and cool the weld to see how it does with that. I wanna introduce vibration to the weld." And if you can show that under vibration, under wide-ranging changes in the environment, that the strength holds up, then by focusing on the strength, which is what you want, you end up with fewer quality problems. But it's turning things around and saying, "Not what I don't want, what do I want?"

0:18:35.3 Andrew Stotz: And...

0:18:36.4 Bill Bellows: And that's what... Go ahead. Go ahead. Andrew.

0:18:37.6 Andrew Stotz: There's two things. The more I think about this quote that you're talking about, to improve quality, don't measure quality, sometimes I think I got it, but sometimes I don't. I just wanna think about a couple of parallels. One of them is sometimes we say in the field of sales and marketing, we may say, "Fill your pipeline and your sales will happen." So focus on the beginning of the process. If you don't have a pipeline of people coming in to your company, into your sales team, there's nobody to sell. So that's an example. We also say sometimes, focus on the inputs and the outputs will take care of themselves. That's another way that we would use something similar. But I'm just curious, what does it mean by "Don't measure quality"?

0:19:25.0 Bill Bellows: Yeah. And that's a good question. I'd say, Taguchi's used to quality being the absence of defects. And quality is what the customer's complaining about. So he's saying, quality problems in terms of don't measure quality, he's saying, "So what are the quality problems?" "Oh, let me tell you, we've got porosity, we've got cracks, we've got drop-through, we've got cracking, cracking of the belt and slipping and the... " This is what people are complaining about. And what he's saying is, the customer's not articulating, "Hey, Andrew, improve the function." They're complaining about the... You just have to interpret that what they're saying is, you have to take where they are. They don't want it to crack. They want it to last longer. They want all these things and say... And the idea is, don't get sucked into what they don't want. Turn it around to, well then, I'm the engineer, and this is what Dr. Taguchi would say, "As an engineer, don't be dumbed down into the complaint world. Turn it around and say, what could you improve? What is the function of that thing you're selling?" And if you improve the function, because again, the beauty of talking about function, if you focus on problems, you eliminate one problem, create another problem, then another problem. Now you're just... And what...

0:21:00.8 Andrew Stotz: So it's whack-a-mole...

0:21:02.3 Bill Bellows: Exactly.

0:21:03.6 Andrew Stotz: It's whack-a-mole in the back end of the process without the awareness of, "What are the customer's needs and how do we understand whether we're hitting the mark?" And...

0:21:12.7 Bill Bellows: Oh, and this is what Dr. Taguchi used to call as whack-a-mole engineering. It's what Ackoff would say, "Today's problems come from yesterday's solutions."

0:21:24.6 Andrew Stotz: So just just to visualize that, can imagine going into a factory and saying, "Look at all these charts and how we reduce the defects of this and that. And this is... " We've reduced all these defects, but in fact, that could be out of touch with what the customer really needs at the end of that production.

0:21:44.1 Bill Bellows: Yes, it is... The beauty is, it is saying... And he would get really angry with people who got sucked into the rabbit hole of eliminating defects, scrap and rework and things like that. And just say... What he's trying to say is, "I want you to be smarter than that. I want you to start to think about what is the function of the machining process? What is the function of the welding process? What is the... " And what was neat was, I spent... On three different occasions, I spent a week with him, watching him engage every day with four teams. A team would come in for two hours, and he would discuss with them whatever the hardware was. I'm not at liberty to say what company it was. [laughter] But it was a really cool company.

0:22:56.9 Bill Bellows: And the people there invited me in because I learned at Dr. Deming's... I attended Dr. Deming's very last four-day seminar, and there met some people that were very close to him. And one of them shared that, there were people for many years, traveled with Dr. Deming. They found out where he was gonna be a given week, maybe called up his secretary Ceilia Kilian and, once he became, bonded when... And somehow Dr. Deming liked you. And then you would say, "Dr. Deming, I'm gonna take a week's vacation next summer. Where are you gonna be in June?" And he'd say, "Well, I'm gonna be at GM corporate headquarters." And what these people told me is that, they would be with him that week, whether he is doing a four-day seminar in Ohio. Now, I don't know who paid for it.

0:23:50.8 Andrew Stotz: No.

0:23:50.9 Bill Bellows: But he gave them access to be with him wherever he was. And one guy told me he was at some high-level GM meetings that week, and he said, "Dr. Deming is there and he and some others." And I think they may have been called "Deming Scholars". I know that term was used. But anyway, this guy was telling me they were there, and this GM executive comes over to him and it says to him, "So, who are you again?" And you say, "Andrew Stotz." And he says something like, "So what might I ask are your qualifications for being here?" And he says, "If Deming overheard that, Deming would turn to the executive, snap at him and say, 'These are my people. What are your qualifications?'" So anyway, inspired by that, I walked out of Dr. Deming's four-day seminar, called up a friend of mine who worked for Dr. Taguchi's company and said, "Deming had people travel with him. I wanna travel with Dr. Taguchi. I don't wanna go to a seminar. I wanna see him in action."

0:24:56.1 Andrew Stotz: Yep.

0:24:56.4 Bill Bellows: And I said, "Can we make that happen?" And it happened, and I got to go inside a company. The lawyers didn't know I was there. And I asked him, I said, to the lawyer, "Do I have to sign anything?" He said, "No. If we let the lawyers know you were here, you wouldn't be here. So, here are the rules. You can't tell anybody what happened, blah, blah, blah, blah, blah." So I get to be a fly on the wall watching him. So, a team would come in and say, "Here's this stuff we're working on." And he would... And they had an approach, which would be, reducing defects or scrap rework. And then he would turn it around for the next hour and a half and get them thinking about function. And after the first week of doing this four times a day for five days, I walked out of there thinking, "There's five basic functions." I started to notice the patterns. And then the second time I did this, a team would come in and I'm thinking, "I know what he's gonna do. He's gonna... He has in mind a function model. And all these things relative to how things come together." And so I did that three times. But, it was neat to get my brain adapted to, "Okay, what's the function? Where's he gonna come? Where's he gonna come?"

0:26:16.0 Bill Bellows: And then I would... The people would present it, and I'm thinking, "I think it's gonna go for function five. Yep. Bingo." So that's what I just wanted to share with the audience tonight. Again, there's a lot of depth. I taught two 40-hour courses at Rocketdyne in Taguchi Methods. So, a 40-hour intro and a 40-hour more advanced. So all I wanted to cover tonight, is that wisdom of not being defect-focused, but for our audience to start thinking about, start to think about the function. In fact, when I was having this conversation with a colleague recently and, 'cause he's talking about turning defect rate, he was thinking turning defect rate data into a loss function. I said, "No, defect rate thinking is acceptability thinking, the loss function is desirability thinking. They don't go together." I said, "What I wanna know is what's causing the defects." And we start diving into what's causing the defects, we can turn it into a variable data as opposed to a discontinuous data. Anyway. And I just wanted to throw out... Go ahead, Andrew.

0:27:34.4 Andrew Stotz: To wrap this up, I'm thinking about, I like what you just said, "Stop being defect-focused." Replace that with...

0:27:44.5 Bill Bellows: What is it we're trying to accomplish?

0:27:47.8 Andrew Stotz: Yeah.

0:27:48.2 Bill Bellows: If you say, "Well, we don't want defects." I know we don't want defects. But what do we want?

0:27:52.9 Andrew Stotz: Do we say replace it with outcome focus, customer focus? What would you say?

0:27:58.1 Bill Bellows: Yeah, well, absolutely it's customer focus. The idea is that, now you start to think in terms of, is what is the greater system in which this is used.

0:28:11.0 Andrew Stotz: Okay. So...

0:28:11.1 Bill Bellows: The defect thinking is just saying it doesn't fit, it doesn't meet requirements. But that doesn't tell me what you're trying to do.

0:28:17.0 Andrew Stotz: Okay. So I think I know what you're saying. Stop being defect-focused, and please walk through door number three.

0:28:25.3 Bill Bellows: Yes! Stop...

0:28:27.7 Andrew Stotz: And in door number three, you're gonna be aware of the customer, the next process, the next flow, the customer of your area and the ultimate customer, and start focusing on the needs and the desires of them, and bring that back in the chain of your process. And you'll be improving, you'll stop being focused on "Fix this, stop this. Don't do that." Let's not have any more of that, and you'll be more into, "Let's do this because this is going to drive a much better outcome, or the exact outcome that our customer wants."

0:29:05.2 Bill Bellows: Yeah, it is, which changes the hat. That may not be the purview of people in the quality organization. So, they're out there counting defects. This is not to say it's their job. Not that they're not in the loop, but it's turning to the people that are more aware... That are more in tune with functionality, which is likely gonna be that people designing the thing, thinking about what's the role of the windshield wiper? Is it to skip across the windshield? Is it to, which is, that chatter. No, we don't want the chatter. So what is it we do want? We want the windshield wiper to move smoothly. And what does that mean? It means at a given second, we want it to be... And this is where the smoothness functionality comes in that I saw Dr. Taguchi many times is, is saying at a given interval of seconds, it should be here, here, here, here, here. And if it does match those positions, then what have we done? We have improved the smoothness of the flow of the wiper blade, or whatever it is that thing.

0:30:21.5 Bill Bellows: And that's the type of thing I'm trying to introduce, in this short episode, people thinking about function, not the lack of quality, but what is it we're trying to achieve? Now, otherwise, we can also say, Ackoff would say, and Dr. Deming would agree with him, is that organizations aren't in business to make a profit. They're in business to do something really well. That's the function of the organization. And then profit is the result of that. As opposed to being profit focus, in which case you start to... You run it as a finance company and misunderstand the focus and you start believing in addition and you end up with a mess.

0:31:04.2 Andrew Stotz: So, let's end it with a cartoon that I saw in The Wall Street Journal. And in that cartoon, it was a couple of guys, young guys wearing suits, and they were talking to each other, and they were either, it was either in an MBA class or they were in a factory or something, and it said, "Things? I don't wanna make things. I wanna make money." [laughter] And the whole point is, money is the result of making great things.

0:31:38.3 Bill Bellows: Exactly. Exactly. Exactly. And that's why... And this is why I so enjoyed about listening to Ackoff, conversations with Russ... Conversations with Dr. Taguchi. And then reading Deming. I don't have any conversation with Dr. Deming and thinking of that there. They, each were astute enough to see the process, the means leading to the result. Tom Johnson would say, "The means are the ends in the making." So you have organizations that are either means-focused, which is process focus, versus, "Did you deliver the report? Did you deliver the thing?" And Dr. Deming's big thing is, by what method? Tom would say, "By what means?" So...

0:32:25.2 Andrew Stotz: All right. Well...

0:32:25.2 Bill Bellows: Anyway, that's what I wanted to expose our audience to tonight.

0:32:29.4 Andrew Stotz: There it is. They've been exposed. Ladies and gentlemen, the exposure has happened. Bill, on behalf of everyone at The Deming Institute, I wanna thank you again for this discussion. And for listeners, remember to go to deming.org to continue your journey. And if you wanna keep in touch with Bill, just find him on LinkedIn. And this is your host, Andrew Stotz. And I'll leave you with one of my favorite quotes from Dr. Deming. And you know this one, you can say it along with me 'cause I say it all the time. People are entitled to joy in work.