Quality, Decision Making and Putting the Human First

Quality stands in a position, sometimes uniquely in an organization, of engaging with stakeholders to understand what objectives and unique positions the organization needs to assume, and the choices that are making in order to achieve such objectives and positions.

The effectiveness of the team in making good decisions by picking the right choices depends on their ability of analyzing a problem and generating alternatives. As I discussed in my post “Design Lifecycle within PDCA – Planning” experimentation plays a critical part of the decision making process. When designing the solution we always consider:

Always include a “do nothing” option: Not every decision or problem demands an action. Sometimes, the best way is to do nothing.
How do you know what you think you know? This should be a question everyone is comfortable asking. It allows people to check assumptions and to question claims that, while convenient, are not based on any kind of data, firsthand knowledge, or research.
Ask tough questions! Be direct and honest. Push hard to get to the core of what the options look like.
Have a dissenting option. It is critical to include unpopular but reasonable options. Make sure to include opinions or choices you personally don’t like, but for which good arguments can be made. This keeps you honest and gives anyone who see the pros/cons list a chance to convince you into making a better decision than the one you might have arrived at on your own.
Consider hybrid choices. Sometimes it’s possible to take an attribute of one choice and add it to another. Like exploratory design, there are always interesting combinations in decision making. This can explode the number of choices, which can slow things down and create more complexity than you need. Watch for the zone of indifference (options that are not perceived as making any difference or adding any value) and don’t waste time in it.
Include all relevant perspectives. Consider if this decision impacts more than just the area the problem is identified in. How does it impact other processes? Systems?

A struggle every organization has is how to think through problems in a truly innovative way. Installing new processes into an old bureaucracy will only replace one form of control with another. We need to rethink the very matter of control and what it looks like within an organization. It is not about change management, on it sown change management will just shift the patterns of the past. To truly transform we need a new way of thinking.

One of my favorite books on just how to do this is Humanocracy: Creating Organizations as Amazing as the People Inside Them by Gary Hamel and Michele Zanini. In this book, the authors advocate that business must become more fundamentally human first. The idea of human ability and how to cultivate and unleash it is an underlying premise of this book.

it’s possible to capture the benefits of bureaucracy—control, consistency, and coordination—while avoiding the penalties—inflexibility, mediocrity, and apathy.
Gary Hamel and Michele Zanini, Humanocracy, p. 15

The above quote really encapsulates the heart of this book, and why I think it is such a pivotal read for my peers. This books takes the core question of a bureaurcacy is “How do we get human beings to better serve the organization?”. The issue at the heart of humanocracy becomes: “What sort of organization elicits and merits the best that human beings can give?” Seems a simple swap, but the implications are profound.

Bureaucracy versus Humanocracy. Source: Gary Hamel and Michele Zanini, *Humanocracy*, p. 48

I would hope you, like me, see the promise of many of the central tenets of Quality Management, not least Deming’s 8th point. The very real tendency of quality to devolve to pointless bureaucracy is something we should always be looking to combat.

Humanocracy’s central point is that by truly putting the employee first in our organizations we drive a human-centered organization that powers and thrives on innovation. Humanocracy is particularly relevant as organizations seek to be more resilient, agile, adaptive, innovative, customer centric etc. Leaders pursuing such goals seek to install systems like agile, devops, flexible teams etc. They will fail, because people are not processes. Resiliency, agility, efficiency, are not new programming codes for people. These goals require more than new rules or a corporate initiative. Agility, resilience, etc. are behaviors, attitudes, ways of thinking that can only work when you change the deep ‘systems and assumptions’ within an organization. This book discusses those deeper changes.

Humanocracy lays out seven tips for success in experimentation. I find they align nicely with Kotter’s 8 change accelerators.

Humanocracy’s Tip	Kotter’s Accelerator
Keep it Simple	Generate (and celebrate) short-term wins
Use Volunteers	Enlist a volunteer army
Make it Fun	Sustain Acceleration
Start in your own backyard	Form a change vision and strategic initiatives
Run the new parallel with the old	Enable action by removing barriers
Refine and Retest	Sustain acceleration
Stay loyal to the problem	Create a Sense of Urgency around a Big Opportunity

Comparison to Kotter’s Eight Accelerators for Change

Measuring Training Effectiveness for Organizational Performance

When designing training we want to make sure four things happen:

Training is used correctly as a solution to a performance problem
Training has the the right content, objectives or methods
Trainees are sent to training for which they do have the basic skills, prerequisite skills, or confidence needed to learn
Training delivers the expected learning

Training is a useful lever in organization change and improvement. We want to make sure the training drives organization metrics. And like everything, you need to be able to measure it to improve.

The Kirkpatrick model is a simple and fairly accurate way to measure the effectiveness of adult learning events (i.e., training), and while other methods are introduced periodically, the Kirkpatrick model endures because of its simplicity. The model consists of four levels, each designed to measure a specific element of the training. Created by Donald Kirkpatrick, this model has been in use for over 50 years, evolving over multiple decades through application by learning and development professionals around the world. It is the most recognized method of evaluating the effectiveness of training programs. The model has stood the test of time and became popular due to its ability to break down complex subject into manageable levels. It takes into account any style of training, both informal and formal.

Level 1: Reaction

Kirkpatrick’s first level measures the learners’ reaction to the training. A level 1 evaluation is leveraging the strong correlation between learning retention and how much the learners enjoyed the time spent and found it valuable. Level 1 evaluations, euphemistically called a “smile sheet” should delve deeper than merely whether people liked the course. A good course evaluation will concentrate on three elements: course content, the physical environment and the instructor’s presentation/skills.

Level 2: Learning

Level 2 of Kirkpatrick’s model, learning, measures how much of the content attendees learned as a result of the training session. The best way to make this evaluation is through the use of a pre- and posttest. Pre- and posttests are key to ascertaining whether the participants learned anything in the learning event. Identical pre- and posttests are essential because the difference between the pre- and posttest scores indicates the amount of learning that took place. Without a pretest, one does not know if the trainees knew the material before the session, and unless the questions are the same, one cannot be certain that trainees learned the material in the session.

Level 3: Behavior

Level 3 measures whether the learning is transferred into practice in the workplace.

Level 4: Results

Measures the effect on the business environment. Do we meet objectives?

Evaluation Level	Characteristics	Examples
Level 1: Reaction	Reaction evaluation is how the delegates felt, and their personal reactions to the training or learning experience, for example: ▪ Did trainee consider the training relevant? ▪ Did they like the venue, equipment, timing, domestics, etc? ▪ Did the trainees like and enjoy the training? ▪ Was it a good use of their time? ▪ Level of participation ▪ Ease and comfort of experience	▪ feedback forms based on subjective personal reaction to the training experience ▪ Verbal reaction which can be analyzed ▪ Post-training surveys or questionnaires ▪ Online evaluation or grading by delegates ▪ Subsequent verbal or written reports given by delegates to managers back at their jobs ▪ typically ‘happy sheets’
Level 2: Learning	Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before to after the learning experience: ▪ Did the trainees learn what intended to be taught? ▪ Did the trainee experience what was intended for them to experience? ▪ What is the extent of advancement or change in the trainees after the training, in the direction or area that was intended?	▪ Interview or observation can be used before and after although it is time-consuming and can be inconsistent ▪ Typically assessments or tests before and after the training ▪ Methods of assessment need to be closely related to the aims of the learning ▪ Reliable, clear scoring and measurements need to be established ▪ hard-copy, electronic, online or interview style assessments are all possible
Level 3: Behavior	Behavior evaluation is the extent to which the trainees applied the learning and changed their behavior, and this can be immediately and several months after the training, depending on the situation: ▪ Did the trainees put their learning into effect when back on the job? ▪ Were the relevant skills and knowledge used? ▪ Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles? ▪ Would the trainee be able to transfer their learning to another person? is the trainee aware of their change in behavior, knowledge, skill level? ▪ Was the change in behavior and new level of knowledge sustained?	▪ Observation and interview over time are required to assess change, relevance of change, and sustainability of change ▪ Assessments need to be designed to reduce subjective judgment of the observer ▪ 360-degree feedback is useful method and need not be used before training, because respondents can make a judgment as to change after training, and this can be analyzed for groups of respondents and trainees ▪ Online and electronic assessments are more difficult to incorporate – assessments tend to be more successful when integrated within existing management and coaching protocols
Level 4: Results	Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee – it is the acid test Measures would typically be business or organizational key performance indicators, such as: volumes, values, percentages, timescales, return on investment, and other quantifiable aspects of organizational performance, for instance; numbers of complaints, staff turnover, attrition, failures, wastage, non-compliance, quality ratings, achievement of standards and accreditations, growth, retention, etc.	The challenge is to identify which and how relate to the trainee’s input and influence. Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training, so they understand what is to be measured ▪ This process overlays normal good management practice – it simply needs linking to the training input ▪ For senior people particularly, annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

4 Levels of Training Effectiveness

Example in Practice – CAPA

When building a training program, start with the intended behaviors that will drive results. Evaluating our CAPA program, we have two key aims, which we can apply measures against.

Behavior	Measure
Investigate to find root cause	% recurring issues
Implement actions to eliminate root cause	Preventive to corrective action ratio

To support each of these top-level measures we define a set of behavior indicators, such as cycle time, right the first time, etc. To support these, a review rubric is implemented.

Our four levels to measure training effectiveness will now look like this:

Level	Measure
Level 1: Reaction	Personal action plan and a happy sheet
Level 2: Learning	Completion of Rubric on a sample event
Level 3: Behavior	Continued performance and improvement against the Rubric and the key review behavior indicators
Level 4: Results	Improvements in % of recurring issues and an increase in preventive to corrective actions

This is all about measuring the effectiveness of the transfer of behaviors.

Strong Signals of Transfer Expectations in the Organization	Signals that Weaken Transfer Expectations in the Organization
Training participants are required to attend follow-up sesions and other transfer interventions. What is indicates: Individuals and teams are committed to the change and obtaining the intended benefits.	Attending the training is compulsory, but participating in follow-up sessions or oter transfer interventions is voluntary or even resisted by the organization. What is indicates: They key factor of a trainee is attendance, not behavior change.
The training description specifies transfer goals (e.g. “Trainee increases CAPA success by driving down recurrence of root cause”) What is indicates: The organization has a clear vision and expectation on what the training should accomplish.	The training description roughly outlines training goals (e.g. “Trainee improves their root cause analysis skills”) What is indicates: The organization only has a vague idea of what the training should accomplish.
Supervisors take time to support transfer (e.g. through pre- and post-training meetings). Transfer support is part of regular agendas. What is indicates: Transfer is considered important in the organization and supported by supervisors and managers, all the way to the top.	Supervisors do not invest in transfer support. Transfer support is not part of the supervisor role. What is indicates: Transfer is not considered very important in the organziaiton. Managers have more important things to do.
Each training ends with careful planning of individual transfer intentions. What is indicates: Defining transfer intentions is a central component of the training.	Transfer planning at the end of the training does not take place or only sporadically. What is indicates: Defining training intentions is not (or not an essential) part of the training.

Good training, and thus good and consistent transfer, builds that into the process. It is why I such a fan of utilizing a Rubric to drive consistent performance.

Management’s Job

In episode 48 of the Deming Len’s podcast, the host refers back to Deming’s last interview, “Dr. Deming: ‘Management Today Does Not Know What Its Job Is‘”

Are You Learning or Just Staying Busy? – In Their Own Words

If you've led an organization or a team for more than a few years, you've probably watched the same problems resurface again and again. Different initiatives. Different people. Same underlying issues. In this episode, John Dues joins Andrew Stotz to dig into a puzzle Dr. Deming pointed to decades ago: "Experience without theory teaches nothing." John unpacks what Deming meant: experience doesn't come with its own explanation. It can answer a question, but only if you're asking one in the first place. Without a theory of why something works, all that hard-won experience just repeats itself. This episode is a great listen for anyone who has ever wondered why years of effort don't always add up to real improvement. TRANSCRIPT 0:00:02.2 Andrew Stotz: My name is Andrew Stotz, and I'll be your host as we dive deeper into the teachings of Dr. W. Edwards Deming. Today, I'm continuing my discussion with John Dues, who is part of the new generation of educators striving to apply Dr. Deming's principles to unleash student joy in learning. The topic for today is why experience isn't enough, and it leads to a pretty interesting puzzle. If experience is such a great teacher, shouldn't businesses and schools and school districts just keep getting better over time? John, take it away. 0:00:42.7 John Dues: Yeah, thanks for having me back, Andrew. Yeah, I think that's exactly the question that got me thinking about this. When I think about my own experience as a school leader and people I've worked with over the years, once you've been, let's say, a principal for 10 or 15 years or a teacher or superintendent, you're a school leader or a teacher leader, you've probably observed hundreds or maybe even thousands of lessons. You've dealt with all kinds of issues, discipline problems, staffing challenges, parent concerns. You've seen curriculum come and go, state assessments change, new initiatives, budget decisions, all kinds of unexpected problems. And yet, I think organizations, complex organizations like schools, you can spend years wrestling with essentially the same challenges year after year, even as the initiatives change and people sometimes change and terminology changes, but a lot of times the underlying problems can remain really, really consistent. And so I started thinking, if experience naturally produces learning, why doesn't all of this experience that gets accumulated reliably produce better organizations? And then I was thinking about something that I read from Deming, who more than 30 years ago said… Probably more than 40 years ago said now, "Experience without theory teaches nothing." 0:02:24.0 Andrew Stotz: This one… That statement always kind of got me when I first heard it. Sometimes I listen to that, "Experience without theory teaches nothing." and think to myself, I know what that means. And sometimes I listen and I think, I don't really know what that means. And so it is kind of a provocative statement because I'm thinking about, well, how did I learn? Did I learn in my past? Are we talking about individual learning or are we talking about learning within an organization? And also, isn't experience the best teacher? So what do you think Dr. Deming meant? 0:03:02.2 John Dues: Yeah, it's like a lot of things I think that he talked about, you sort of have to stop and think. And even after you've thought about it once or twice, you can come back to it and kind of question yourself, "Did I really understand that?" You might have to go talk to somebody. But when I first read it, truth be told, as I was learning this stuff, I thought he had to be exaggerating. I thought the same thing: how could experience teach us nothing? I mean, obviously, like you were saying, we learn things from experience. But I think my interpretation is that Deming's point is subtler than that. I don't think he's saying, or he wasn't saying, that experience has no value. I think what he was saying is that experience doesn't come with its own explanation. Something happens in our classroom or our school and we observe it, but what does that experience mean? What should we conclude from it? What should we do differently next time? I don't think that experience can answer those questions by itself. And in 'Out of the Crisis', Deming wrote, "Experience will answer a question, and a question comes from theory." And I think that's really the key. I think experience becomes useful when we're using it to answer a question about how we think the world works. Again, in our classroom, our school, our school district, that type of thing. 0:04:27.7 Andrew Stotz: Imagine that you're a kid walking down the street and a pit bull comes running at you and bites you on the arm. It's not too serious. The owner manages to get them off you. There's an experience. And it's experience tied with an emotion, which means it's memorable. You'll never forget that. What could you learn from that? Can you learn from that? Does what you learn from that apply to other dogs? It just, it opens up a Pandora's box that I like to think about in that particular case. 0:05:11.8 John Dues: Well, and I think in a case like that, you go through an experience like that, you're definitely gonna have a theory the next time you run into a dog, what you're gonna do, right? Especially if it looks like a… 0:05:23.4 Andrew Stotz: Yeah. 0:05:23.5 John Dues: Yeah. For sure. 0:05:24.4 Andrew Stotz: And this word theory, most of the time we don't really use that in our daily life, but we have words like theory, hypothesis, guess, hunch. So what exactly do you think he's talking about when he says theory? 0:05:45.9 John Dues: Yeah, I think that that can be, for if you're not in academia, I think that that can maybe be one of the biggest barriers to understanding this idea. I think, kind of like what you're saying, when people hear the word theory, you often are imagining something academic or something abstract. But if you think about it, we're using theories constantly, no matter what our work is. And when I'm saying theory, I'm just talking about an explanation for how and why something works. So, really every decision I'm making as a leader rests on some theory. I'm thinking about, if I'm a principal and I decide that something like weekly classroom observations will improve instruction, there's a theory behind that decision. I'm saying really, by putting weekly classroom observations into my practice, I'm saying if we increase the frequency of classroom observations and feedback, then teaching will improve. Right? That's my theory. If a superintendent who's leading his district and they have some issues with reading and they purchase a new reading curriculum, there's a theory, if teachers use this curriculum, student reading outcomes will improve. And this could apply to any number of things. The leadership team of a school district changes the schedule, they restructure what staffing looks like across their schools. You add an intervention period to your school building, the state changes the teacher evaluation system, or the state launches an attendance initiative in response to this chronic absenteeism problem we've been seeing. Every one of these decisions reflects a theory about cause and effect, basically. 0:07:46.5 Andrew Stotz: And there's two things about theory that I like to do when I'm teaching. I like to ask a question that we could prove what the answer is. And let's just say, how many companies in the world have a strong level of profitability and growth relative to peers sustained over a five-year period? That's a very specific question and I have very clear definitions of all of that. And out of… Then I ask the audience, my students, out of 8,500 companies in the sample that I have, how many companies do you think have sustained this outperformance over five years? And then they make guesses as to what they think the answer is, and then I walk them through, here's how I tried to prove it. Now, you could argue that my… You'd define something differently or you'd do the test differently over three years or seven years. One student said, "Why not 10 years?" All of that. But in the end, I say, "Okay, my answer to that question is 400." 0:08:59.9 John Dues: Interesting. 0:09:01.0 Andrew Stotz: And so then that leads me to another step in the process. And what I'm trying to teach my students in Valuation Masterclass Bootcamp is the idea of how do you think like an analyst? An analyst says they come up with a theory or an idea, like, "I think some stock, stock ABC, is gonna double in the next six months." Okay, that's a theory, and you're gonna put money down on that theory. You may try… You need to try to support it with evidence and all that, and then in six months, if you didn't write it down, you'll probably forget it or you'll misremember it. 0:09:37.7 John Dues: Yep. 0:09:38.3 Andrew Stotz: So the question isn't really whether leaders have theories. I think we all have theories, and we're acting on these theories. So where do we go from there? 0:09:49.9 John Dues: Yeah, I think what you just said is exactly right. The question is, I think, whether those theories are explicit enough to examine. So you talked about writing it down, that's a common misstep. We don't write our theories down. I think most of our theories are implicit. We don't say, "Here's my theory about how this system works. Here's my theory about how this classroom works. Here's my theory about why weekly classroom observation will work to improve instruction." We just sort of act, and then something happens. Sometimes the results improve, sometimes they get worse, sometimes nothing really changes. Probably a lot of the time it's that. And then typically what we do is tell ourselves a story about why that outcome happened, whichever one it is, right? And that's, I think, where experience becomes particularly dangerous, because two people can observe exactly the same event and draw completely different lessons from that. 0:10:52.2 Andrew Stotz: And my lesson is the correct one. So give me an example of when you've seen that? 0:11:01.9 John Dues: Yeah, I mean, I think, imagine a school that introduces a new instructional strategy and then test scores rise the following year. It'd be very easy for that principal to say, "The strategy worked. Now let's expand it." Another might say, "Well, wait a second. The tea… Or sorry, the students were simply a stronger cohort that particular year." Someone else might say, "Well, remember, we also changed how we staffed that particular school last year." Another might say, "Now, I just noticed that the state test that we were looking at actually was easier this year than in previous years." Everybody has access to that same, the same instructional change and then the same exact outcome, but they're interpreting that experience through different theories about what caused the results. So, in an example like that, experience doesn't settle the question. Theory determines what questions we ask of the experience. 0:12:16.1 Andrew Stotz: So once you start looking at organizations this way, you say you began seeing theory everywhere? So what do you mean by that? 0:12:25.5 John Dues: Yeah, I mean, I think once you start to think in this way or see in this way, you almost can't unsee it. It's like a lot of things with the Deming philosophy. For example, a strategic plan. We talked about that last time. A strategic plan, the whole plan in and of itself is a theory. It basically says, "We believe these priorities and these actions will move us toward this aim or outcome." A budget is a theory. In that case, we're saying, "We believe putting resources here rather than there will produce greater value." Our approach to professional development, it's a theory about how adults learn and how changes in adult knowledge will affect student outcomes. Another example would be teacher evaluation systems contain theories about motivation and improvement. Accountability systems contain theories about how performance changes. Hiring systems contain theories about what predicts effectiveness. The point here isn't that any particular theory is necessarily right or wrong. The important thing is that these theories already exist. They're guiding our behavior whether we've made them visible or not. And I think the main point here is that if the theory is invisible, then it's very, very difficult to improve it. 0:13:48.2 Andrew Stotz: And you connect this idea to your own experience with strategic planning, which we've talked about a lot at United Schools. So how did your theory change? 0:13:59.7 John Dues: Yeah, I think I kind of mentioned this in the last episode, but several years ago, my thinking about strategic planning really started to evolve and change quite a bit. We had written strategic plans a number of times before. We've often done it with an outside consultant. And we had done that about a decade ago, and it wasn't a bad plan. It had good priorities, useful ideas, and those types of things. But once it was kind of finalized, increasingly as we got away from the point where it was a new plan, it just kind of really quickly became disconnected from the way we actually manage the organization. But I think fundamentally the problem wasn't the plan itself. Our plan reflected our theory at the time about what strategic planning was supposed to do. And our basic theory was something like, you go through a planning process, you identify your priorities, you create the plan, and then you implement it. And basically over time, I've stopped believing that that's an effective approach basically. 0:15:10.1 Andrew Stotz: Yeah. I mean, one of the things that goes on in my head is the, what's the difference between strategy and planning? And now after doing a lot of work on strategy, strategy is some element that we think is something that could really… It's a pivotal point of what in our business we can change to be able to affect a significant outcome. The plan is kind of how we implement that. And that's where you see a lot of plans are very weak because they're just priorities and plans, but the clear vision of where you're going and the unique way that you're gonna break this down and implement it is something that I think a lot about. So you've talked about how your idea has changed over time. Tell me what's changed? 0:16:05.4 John Dues: I think what I eventually realized was that we didn't just need this document that we put a bunch of time and effort into. And it did describe where we wanted to go, but really what we needed to do was we needed the plan to make our thinking visible. We needed it connected to the way we actually manage the organization. And we needed it to be capable of changing as we learned. So that's a delicate balance there, right? And basically we were changing how we thought about what happened after the plan was implemented. We really needed a management system. It affected how we ran our leadership team meetings, it affected how we reviewed performance, it affected how we connected our long-term strategy to annual goals and smaller quarterly priorities and then down to the daily work. So the important learning wasn't that we finally figured out how to write a better strategic plan. It wasn't that. The deeper learning was we changed our theory about what strategic planning was for. And I think that's really the distinction that's most important here. 0:17:19.1 Andrew Stotz: So you're connecting planning with PDSA. So talk to me a little bit about that, because I think this may be the most important part of what you're talking about. 0:17:34.0 John Dues: Yeah, I think they're closely connected. So if organizations like a school system are gonna learn from experience, they need a disciplined way to do sort of three things. First, what we've already talked about, we got to make the theories explicit. The big theories or the main theories that are gonna guide our work, we got to write them down, make them explicit, let everybody in our organization know that they're there. And then second, we have to have a way to test those theories against reality. And then third, we have to revise the theories based on what they learn. And I think that's really where planning, something like the plan within the strategic plan and something like the Plan-Do-Study-Act cycle, really fit together. In either case, planning exists to make theory explicit. But when we create a plan, what we're really doing is putting our current thinking on the table. We're saying, "Here's our aim, here's what we believe matters, here's what we're gonna do, and here's what we expect will happen." And I think a good plan makes all of those assumptions visible to the entire team. 0:18:42.4 Andrew Stotz: So to the listeners out there, the viewers, ask yourself, "Am I making my theory explicit?" You know, that's one of the things I… We're in the process of testing a new strategic move for my coffee business. And we've pulled together a team of the best qualified people to start to test it. And we've definitely written out a lot, but now, just listening to you, I realize that I think we need to do more to make our theory very explicit so that we don't drift also into just doing the work, that we really are trying to make a strategic move that's gonna make an impact. So when we think about plan and making theories explicit, so tell me about how PDSA will test that? 0:19:38.8 John Dues: Yeah, I think that's how I'm kind of thinking about this. Your strategic plan has your priorities, your goals, and maybe even your quarterly actions. But then where your PDSA comes in, the PDSA cycle, it's basically there to then test and improve that theory. So you know PDSAs well, and probably a lot of the Deming community does as well. You make a prediction, we test a change, we observe what actually happens, and then we compare the prediction with the result. And the difference between those things is really where all the value lies, because that's where the learning occurs. So the goal of PDSA isn't merely to answer, did it work? That's often too simplistic. The better question is, what did this test teach us about our theory? Maybe the result matched the prediction, maybe it didn't, but either way, it generated new knowledge for us to then influence the next prediction, the next test, and eventually the plan itself. 0:20:40.7 Andrew Stotz: So what you're saying is that planning and PDSA are really parts of the same system. And what I'm thinking about myself is that I'm working with my accounting team on PDSAs. And you can say it's a very small and narrow part of our business that they're working on because they don't have the influence across the whole business. And they say that to me, and I understand that. So we focus in on what they can change and what testing out what that would mean. And then… But at the corporate level, at the senior level, it is this strategic planning. So maybe you can explain a little bit more. 0:21:17.6 John Dues: Yeah, the way I have started to think about it is that planning, strategic planning, and PDSA are really parts of the same system. So planning makes our current theory explicit, and then PDSA is what we do to improve that theory over time. So sort of put together, they form the learning system within an organization. So again, the plan sort of represents our best current understanding of how we're gonna achieve our aim. And then we actually meet reality. We start to act. We make predictions, we test the changes, we study what happens, we revise our understanding, and eventually that revised understanding gets reflected back in the plan. So you get into this sort of continual cycle, aim, current theory, action, PDSA, learning, revised theory, updated plan, and then that cycle just sort of continues on and on forever, right? And that's a very different conception of leadership from simply creating plans and implementing. Leadership becomes the work of continually improving the organization's understanding of how its system works, basically. 0:22:33.7 Andrew Stotz: Yeah, so I wanna repeat two things that you said. Planning makes our current theory explicit and PDSA improves that theory. That, those are the keys. And one of the things to think about is organizations accumulate activity, I believe is what you're saying, much faster than they accumulate knowledge. And that could be your explanation why they're not improving or improving at the pace that they could be. 0:23:04.2 John Dues: Yeah, I think this is probably true in the vast, vast majority of organizations. This idea of organizations accumulating activity much faster than they accumulate knowledge. And just think about the organization that you've been a part of or observed in. Think about everything that's going on day-to-day, the meetings, the initiatives, the training and professional development, assessments, planning sessions, data reviews, consultants, new programs, new technology, new policies. There is no shortage of activity. I know that's for sure true. This would resonate with 100% of educators, I am sure of that. But the thing is that activity and learning are definitely not… They're not the same thing, so… And that brings us sort of the point back to the intro. You can run the same process year after year without getting any better at it. You can hold hundreds of data meetings without improving your theory about what produces better outcomes. You can implement initiative after initiative, and this is what school systems do without learning very much about the underlying system. So I think the question for leaders isn't just "What did we do?", it's "What did we learn that changes what we believe about how the system works?" And if you can't answer that question, I think what you've done is accumulated experience without accumulating much knowledge. And again, I think this is probably the sort of typical way most school systems operate. 0:24:35.7 Andrew Stotz: Congratulations. You've run the same process year after year without substantially improving it in any way. So how does this change the way you think about what's the leader's job? Because everybody's doing all these things. [overlapping conversation] 0:24:51.2 John Dues: Exactly. And I'm… Over time… Yeah, I mean, at different times, I've been just as guilty of this as the next leader. But it does completely sort of change how I think about the leader's job. Earlier in my career, I thought what leadership was was largely about knowing exactly what to do. You gain experience, you develop expertise, you get better at recognizing patterns, and then eventually you're in the leadership chair and you're the person who has all the answers. And I think a lot differently now. I think experience matters. Expertise matters, clearly. But I think the leader's job isn't simply to accumulate answers. The leader's responsibility is to build an organization capable of learning. And that, again, goes back to making those theories explicit. I think a big part of this is making predictions a part of how you operate every day. Testing and experimenting should be how you operate every day, studying the variation in your system, making it safe to discover that our assumptions were wrong, and then we sort of model revising our thinking. So the goal isn't to prove that leadership was right, which is sometimes what's happening in places. The goal is to continually improve the theory that's guiding the organization. 0:26:19.8 Andrew Stotz: That's quite a challenge. If someone listening remembers just one thing from this conversation, what would you say? 0:26:28.1 John Dues: Yeah, I mean, I think one, turning back to that original quote from Deming, "Experience without theory teaches nothing." And I'm sure people that have studied Deming will recognize that's the first part of a longer sort of section that goes on about this particular idea. But again, most organizations already have an enormous amount of experience. What they often lack is that disciplined system for turning that experience into knowledge. I think every important leadership decision rests on some theory about how the system works. So you have theories. You just do. That's just a fact. The question is whether you can make those theories visible enough to test them. And that's why I've come to believe that organizations don't improve simply by accumulating experience. They improve when they systematically improve the theories that guide their actions. And I think planning can help make the theories explicit and then PDSA can help us test and improve them. And so together, I think that that's the way organizations will then be able to continually learn towards… Learn their way towards their ultimate aim, their purpose. 0:27:42.8 Andrew Stotz: Yep, that's a great wrap-up. And I think if I had to say one thing, I've taken a lot of notes here and I would say what I like is that planning makes our current theory explicit and PDSA improves that theory. And that just, that loop to me is a big takeaway. So I appreciate that. 0:28:10.3 John Dues: Yeah, it was a great discussion. 0:28:12.3 Andrew Stotz: Yeah. So on behalf of everyone, of course, at The Deming Institute, I wanna thank you again for the discussion. And for listeners, remember to go to deming.org and jump into DemingNEXT to continue your journey. You can find John's book, 'Win-Win: W. Edwards Deming, the System of Profound Knowledge, and the Science of Improving Schools' on amazon.com. This is your host, Andrew Stotz, and I'll leave you with one of my favorite quotes from Dr. Deming, and that is, "People are entitled to joy in work."

I’ve written recently about driving fear out of the organization. Without a doubt I think this is the number one task for us. True North for the quality profession.

The source of innovation is freedom. All we have—new knowledge, invention—comes from freedom. Somebody responsible only to himself has the heaviest responsibility. “You cannot plan to make a discovery,” Irving Langmuir said. Discoveries and new knowledge come from freedom. When somebody is responsible only to himself, [has] only himself to satisfy, then you’ll have invention, new thought, now product, new design, new ideas.
Dr. W. Edwards Deming

Whataboutism

Whataboutism is the common term for a version of the tu quoque fallacy, a diversionary tactic to shift the focus off of an issue and avoid having to directly address it by twisting criticism back onto the critic and in doing so revealing the original critic’s hypocrisy.

Whataboutism often results in a comparison of issues as pure deflection. We see it when individuals are always focused on why others get ahead and they don’t, looking for comparisons and reasons they are being treated unfairly instead of focusing on their own opportunities for improvement. It is so easy to use when we are faced with criticism, “Well, what about … ?”

We also see whataboutism in our cultures. Maybe it is a tendency to excuse your own team’s shortcomings because obviously the sins of another team is so much worse. This is a result of, and strengthens, silo-thinking.

Building the feedback process to reduce and eventually eliminate whataboutism is critical.

Transparency

Transparency in organizations generally falls into one of two categories: proactive and demand-driven.

Proactive dissemination refers to information that the organization’s leadership makes known about its activities and performance. Practical expressions can range from strategic and tactical goals, third-party evaluations, and organizational risks.
Demand-driven access refers to an organizational commitment to respond to employee requests for specific kinds of information or documents which otherwise would not be accessible.

The idea of transparency can also be unpacked in terms of its directionality. Disclosure cuts both ways, channeling information upwards as well as downwards in the organization. Leadership sharing insight into decisions is an example of downwards, where idea management processes are upwards.

The type of transparency also matters:

Opaque or fuzzy transparency involves the dissemination of information that does not reveal how organizations actually behave in practice, whether in terms of how they make decisions, or the results of their actions. The term also refers to information that is divulged only nominally, or which is revealed but turns out to be unreliable.
Clear transparency sheds light on organizational behavior, which permits interested employees to pursue strategies of constructive change.

This distinction between clear and opaque is grounded on the premise that if transparency practices are going to meet their goals of transforming organizational behavior, then they must be explicit in terms of who does what, and who gets what.

Transparency requires not only a willingness, but processes to drive it.