The relationship between return on investment and quality of study methodology in workplace health promotion programs

Objective: To determine the relationship between return on investment (ROI) and quality of study methodology in workplace health promotion programs. Data Source. Data were obtained through a systematic literature search of National Health Service Economic Evaluation Database (NHS EED), Database of Abstracts of Reviews of Effects (DARE), Health Technology Database (HTA), Cost Effectiveness Analysis (CEA) Registry, EconLit, PubMed, Embase, Wiley, and Scopus. 

Study Inclusion and Exclusion Criteria: Included were articles written in English or German reporting cost(s) and benefit(s) and single or multicomponent health promotion programs on working adults. Return-to-work and workplace injury prevention studies were excluded. 

Data Extraction: Methodological quality was graded using British Medical Journal Economic Evaluation Working Party checklist. Economic outcomes were presented as ROI. Data Synthesis. ROI was calculated as ROI = (benefits - costs of program)/costs of program. Results were weighted by study size and combined using meta-analysis techniques. Sensitivity analysis was performed using two additional methodological quality checklists. The influences of quality score and important study characteristics on ROI were explored. 

Results. Fifty-one studies (61 intervention arms) published between 1984 and 2012 included 261,901 participants and 122,242 controls from nine industry types across 12 countries. Methodological quality scores were highly correlated between checklists (r =.84-.93). Methodological quality improved over time. Overall weighted ROI [mean 6 standard deviation (confidence interval)] was 1.38±1.97 (1.38-1.39), which indicated a 138% return on investment. When accounting for methodological quality, an inverse relationship to ROI was found. High-quality studies (n = 18) had a smaller mean ROI, 0.26±1.74 (.23-.30), compared to moderate (n=16) 0.90±1.25 (.90-.91) and low-quality (n=27) 2.32±2.14 (2.30-2.33) studies. Randomized control trials (RCTs) (n = 12) exhibited negative ROI, -0.22±2.41(-.27 to -.16). Financial returns become increasingly positive across quasi-experimental, nonexperimental, and modeled studies: 1.12±2.16 (1.11-1.14), 1.61±0.91 (1.56-1.65), and 2.05±0.88 (2.04-2.06), respectively. 

Conclusion: Overall, mean weighted ROI in workplace health promotion demonstrated a positive ROI. Higher methodological quality studies provided evidence of smaller financial returns. Methodological quality and study design are important determinants.