Got measures? Evaluation approaches and challenges

By David K. Ahern, RWJF Aligning Forces for Quality.

On March 26^th, I presented via tele-video conference on approaches to evaluation for the Project HealthDesign grantees who were participating in a workshop at the Center for Better Health at Vanderbilt.  Despite some trepidation about the technology not working, it seemed to go well.  I think we will use it more often in our work with the RWJF Aligning Forces for Quality program going forward.  

The focus of the talk was to promote both qualitative and quantitative methods appropriate for the type, scope, and nature of the projects.  It was a lively session that included an exercise for the grantees to generate and discuss possible evaluation approaches to a hypothetical project scenario.  The biggest challenges that emerged were: (1) measures that would be cross-cutting and (2) integration of ODLs into the clinical enterprise. 

With respect to measures, in addition to including condition-specific instruments relevant to each of the individual projects, patient and provider engagement and activation received much attention.  I suggested Judy Hibbard's Patient Activation Measure (PAM) as one option that seemed to make sense across all projects since one goal is to engage or activate patients to monitor and report ODLs. What other measures are available to capture this critical dimension?

The questions of “Which providers will manage the ODLs” and “What will these providers do with the stream of ODLs” were another topic of discussion.  Assuming we can address these fundamental questions then the need for a measure emerges.  How can we assess adoption and uptake by providers of these new data streams?  

Comments

[Judy Hibbard]

David mentioned in his post the idea of using the PAM as an outcome measure in the Project Health Design evaluation. The PAM has been used as an outcome measure in multiple studies. An advantage for this evaluation is that it would provide a consistent metric across the grantees. It is possible that the more activated are the ones who are more likely to track or consistently track ODLs. It is also likely that the experience of tracking ODLs will increase participants' activation scores. It would be useful to examine these questions, as well as whether different types of supports, including provider support, are linked with increases in activation. We do know from other studies that increases in activation are associated with improvements in multiple health behaviors.