A significant change has occurred in health care with increasing reliance on evidence-based guidelines in the performance of medical and behavioral treatments. The standards used to evaluate medical treatments are usually straightforward: randomized, controlled trials (RCTs) can be conducted to determine if changes occurred before or after treatment. Changes before and after taking a pill, for example, can usually be easily accessed. Consistent results in conducting similar results is what is used to set up evidence-based guidleines. When it comes to assessing educational and behavioral interventions, however, you not only need to access acquisition of skills, but also if and how they are performed. You thus have the complexity of measuring two changes: learning and behavioral. As self-management skills need to be used across time in the management of chronic illness, the matter becomes even more complex. Lenz and colleagues pointed out that common methodologies used in systematic reviews do not allow for adequate appraisal of complex interventions. They described and critically look at current methods of systematic reviews on complex interventions, using diabetes and hypertension patient education as examples. They searched several sets of reviews, including those found in the Cochrane Library. Systematic reviews focusing on diabetes or hypertension patient education were included. Two investigators independently evaluated the reviews, with evidence of three patient education programs of diabetes and hypertension self-management implemented in Germany used as a reference. The authors included 14 reviews in their summary.

COMMENTS: Lenz and coworkers concluded that methods of current systematic reviews are not fully equipped to appraise patient education and self-management programs. Since these are complex and heterogeneous interventions, consideration of aggregated evidence is necessary. We concur completely with the authors in believing that systematic reviews are of little value in evaluating complex interventions, including self-management. Treating behavioral change that occurs over time is not identical to taking a pill. Systematic reviews of most medical treatments are highly useful, but these topics are not as complex as assessing learning and behavioral changes. As a result, reviews such as the Cochrane Library offer only sterile conclusions that have little value in helping those wishing to create and evaluate self-management programs for chronic illnesses, including diabetes and hypertension.

M. Lenz et al. Meta-analysis does not allow appraisal of complex interventions in diabetes and hypertension self-management: a methodological review. Diabetologia, May 23, 2007.