A modified Delphi study to identify the features of high quality measurement plans for healthcare improvement projects

Why it matters

Measuring quality of care over time is essential for monitoring systems, assessing progress, generating feedback and driving improvement for patients. But measurement for improvement is challenging for healthcare improvement teams.  One risk is that teams may sometimes resort to poorly validated measures, and collecting and analysing the data isn’t always done to a high standard. Local ownership of measurement may help teams to engage with and commit to measures, but does not remove these risks.

Guidance that explains “what good looks like” in measuring for improvement might have value in addressing these problems. In particular, better planning for measurement may enable teams to avoid some common mistakes.

This paper describes a consensus-building process to identify which features are important in a quality improvement measurement plan.

Our approach

We used a three-stage consensus-building approach. First, we reviewed the literature to identify important features of measurement plans, and framed each feature as a question.

Next, we conducted a two-round online Delphi exercise, bringing together individuals experienced in developing measurement plans for improvement initiatives with experts in the study of improvement. The participants considered each other’s views in reaching agreement.

Then, we worked with the participants to finalise the list of features.

What we found

• We generated a list of 74 questions that could help quality improvement teams to identify important features of measurement plans.
• The questions fell into five categories: design of measurement, data collection and management, analysis, action, and embedding. Those designing, implementing, and evaluating quality improvement interventions should be able to use the questions to help create transparent and complete measurement plans.
• The list of questions is long, reflecting the complexity of measuring improvement. Further work will be needed to understand which questions are most important, and to test their feasibility and usefulness.
• Although the study helped to identify features of a good measurement plan and whether the features are articulated transparently, it did not address the issue of appropriateness of methods. This may also be a focus of future work.

Related content from our open-access series, Elements of Improving Quality and Safety in Healthcare