studying failure and the checklist approach
How this happened –it involved a checklist, of course– is instructive. But first think about what happens in most lines of professional work when a major failure occurs. To begin with, we rarely investigate our failures. Not in medicine, not in teaching, not in the legal profession, not in the financial world, not in virtually any other kind of work where the mistakes do not turn up on cable news. A single type of error can affect thousands, but because it usually touches only one person at a time, we tend not to search as hard for explanations.
Sometimes, though, failures are investigated. We learn better ways of doing things. And then what happens? Well, the findings might turn up in a course or a seminar, or they might make it into a professional journal or a textbook. In ideal circumstances, we issue some inch-thick set of guidelines or a declaration of standards. But getting the word out is far from assured, and incorporating the changes often takes years.
That is from The Checklist Manifesto, by Atul Gawande. In the weeks since I’ve finished the book, I find myself referring to it often in completely unrelated conversations.
What I’ve wondered most is how a checklist approach might improve the work I do, with respect to both research and teaching. Anyone who has ever worked with me knows what a big fan I am of to-do lists, which I think is distinct from the checklist approach that Gawande writes about. Whereas my to-do lists remind me of things I need to get done, the checklist approach entails having a standardized system of action/response, whether to an emergency (like those pilots potentially face), or to a procedure with high risks (like those surgeons face). For conducting research, I don’t know that we can create a standardized checklist. Anyone disagree? I also don’t know that we study our research failures enough. Sure, people have talked a lot about the vaccine-autism “research” fiasco, but might we academics say that’s an atypical case of research failure? The kind of failure I’m talking about are the studies that don’t get published because of null findings, or worse yet, because the research wasn’t carried out in a way that would generate useful data to even get null findings. As someone who often collects data in faraway places, it would be instructive to learn about failed research projects and pitfalls to avoid.
The teaching part of my job seems much more conducive to the checklist approach. I haven’t used a checklist, but in preparation for the upcoming fall semester, I’ve come up with a few things to put on a checklist: have students write every day in class, give students the opportunity to ask questions before moving on to new topics, make sure students understood the readings even if they won’t be discussed in lecture. Are there other things students/scholars would recommend? I have students fill out evaluations after each exam (for them to give feedback on/evaluate the course, but also for me to evaluate what they’ve retained after the exam is over), and these have helped me redirect efforts based on the respective class’s needs/style.