Skip to content

studying failure and the checklist approach

4 August 2011

"Q400 Checklist" shared by hotelcoffee via cc license on Flickr

How this happened –it involved a checklist, of course– is instructive. But first think about what happens in most lines of professional work when a major failure occurs. To begin with, we rarely investigate our failures. Not in medicine, not in teaching, not in the legal profession, not in the financial world, not in virtually any other kind of work where the mistakes do not turn up on cable news. A single type of error can affect thousands, but because it usually touches only one person at a time, we tend not to search as hard for explanations.

Sometimes, though, failures are investigated. We learn better ways of doing things. And then what happens? Well, the findings might turn up in a course or a seminar, or they might make it into a professional journal or a textbook. In ideal circumstances, we issue some inch-thick set of guidelines or a declaration of standards. But getting the word out is far from assured, and incorporating the changes often takes years.

That is from The Checklist Manifesto, by Atul Gawande. In the weeks since I’ve finished the book, I find myself referring to it often in completely unrelated conversations.

What I’ve wondered most is how a checklist approach might improve the work I do, with respect to both research and teaching. Anyone who has ever worked with me knows what a big fan I am of to-do lists, which I think is distinct from the checklist approach that Gawande writes about. Whereas my to-do lists remind me of things I need to get done, the checklist approach entails having a standardized system of action/response, whether to an emergency (like those pilots potentially face), or to a procedure with high risks (like those surgeons face). For conducting research, I don’t know that we can create a standardized checklist. Anyone disagree? I also don’t know that we study our research failures enough. Sure, people have talked a lot about the vaccine-autism “research” fiasco, but might we academics say that’s an atypical case of research failure? The kind of failure I’m talking about are the studies that don’t get published because of null findings, or worse yet, because the research wasn’t carried out in a way that would generate useful data to even get null findings. As someone who often collects data in faraway places, it would be instructive to learn about failed research projects and pitfalls to avoid.

The teaching part of my job seems much more conducive to the checklist approach. I haven’t used a checklist, but in preparation for the upcoming fall semester, I’ve come up with a few things to put on a checklist: have students write every day in class, give students the opportunity to ask questions before moving on to new topics, make sure students understood the readings even if they won’t be discussed in lecture. Are there other things students/scholars would recommend? I have students fill out evaluations after each exam (for them to give feedback on/evaluate the course, but also for me to evaluate what they’ve retained after the exam is over), and these have helped me redirect efforts based on the respective class’s needs/style.

3 Comments leave one →
  1. A. Ross permalink
    11 August 2011 10:29 pm

    A checklist seems to me to be distinct from a to-do list in that each step builds on the last. An error in one stage of the progression can cause problems later, while if you skip one item on your to-do list, it usually doesn’t affect the other items to be done.

    Thinking of a checklist in this way, I personally consider the scientific method as a very broad checklist in research. It provides a template for how research should progress. And having done fieldwork numerous times with a specific type of instrument (surveys of local government officials), I have honed that method into my own.

    Briefly reflecting on my recent work, I could identify places where I didn’t fully complete (or do in the most ideal or correct way) a step. For example, I recognize now that one of my survey questions was not worded in a way to measure the concept I wanted to capture. Because this part of the “checklist” (in the progression of my version of scientific method) was not completed fully, the subsequent tasks suffer a bit. I can take that as a lesson of how to further refine my “checklist” and make sure that next time, I do it better.

  2. Candace Miller permalink
    29 August 2011 9:46 am

    Thanks for the book recommendation. Finished it last night and looking at life through the lens of a check list now. Only change so far is not forgetting anything when leaving house / traveling with family (garbage emptied, mail held, air conditioning/heat off or down, baby bag filled, ….) 😉


  1. Fall « haba na haba

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: