The Burden of Measurement
Two weeks ago, Dr Robert Wachter wrote an op-ed piece in The New York Times, “How Measurement Fails Doctors and Teachers” (http://www.nytimes.com/2016/01/17/opinion/sunday/how-measurement-fails-doctors-and-teachers.html) .
93 readers offered comments before the commentary cut-off by the site. Many were first-hand accounts by doctors and teachers about measurement that reduces time for direct interaction with patients or students and worse, generates data that may be of dubious relevance and quality.
As someone educated in statistics and active in quality improvement applied to healthcare for the past 16 years (and as a former high school math teacher), the article and comments offer a sobering perspective.
Just as experiments in the physical or chemical sciences have a different support for generalization than experiments in social settings (blogs here and here and here), I sympathize with Dr Wachter’s assertion that the activities of medicine and teaching are different than the business of consumer products like cars and smartphones. As is often the case in measurement of system performance important to people, we need to find measures that are just “good enough” to tell us whether we are getting closer to our aim.
People like me have to help those doing the work of healing patients and educating students find meaningful measures that yield more benefit than cost. While the benefit-cost trade-off itself is subject to debate and illusory attempts at precision, people of good-will and ordinary intelligence can almost always describe measurement ideas worth considering. Then we measurement folks have the obligation to turn those descriptions into feasible measurements, checking back frequently with the professionals who will use them—applying our own sermons about continuous improvement and driving out waste to the measurement system itself.
(A related objection to the problems of measurement in medicine was published the same week as Wachter’s piece, in the NEJM perspective “Medical Taylorism” by Drs Pamela Hartzband and Jerome Groopman, NEngl J Med 374;2, January 14, 2016, 106-108, http://www.nejm.org/doi/full/10.1056/NEJMp1512402. Hartzband and Groopman go beyond measurement issues, challenging the universal application of Toyota-inspired Lean methods to health care. Spirited rebuttals that describe a people-centered application of scientific management in general and Toyota Production System perspective specifically may be found in the comments to the NEJM paper on their site, as well as more extensively here, here and here.)
