The aim of this study is to systematically review and critically assess analytical observational epidemiology studies investigating the association between levels of drinking water hardness and cardiovascular disease. We searched electronic databases and used standardised forms to extract data and assess study quality. Of 2,906 papers identified, 14 met the inclusion criteria (nine case control and five cohort studies). Of the nine case control studies, seven examined both drinking water magnesium and calcium and risk of death from cardiovascular disease. A pooled odds ratio showed a statistically significant inverse association between magnesium and cardiovascular mortality (OR 0.75 (95%CI 0.68, 0.82), p < 0.001). Only two studies reported a statistically significant effect for calcium. Substantial heterogeneity between studies made calculation of a summary estimate for drinking water calcium inappropriate. Of three cohort studies reviewed, two were of good quality. A weak suggestion that soft water was harmful in females and possibly associated with a slightly greater risk of sudden death was reported, but there was no association between water hardness and mortality from stroke or cardiovascular disease. This study found significant evidence of an inverse association between magnesium levels in drinking water and cardiovascular mortality following a meta-analysis of case control studies. Evidence for calcium remains unclear.