In this paper, we present an adaptive gradient based method to restore images degraded by the effects of both noise and blur. The approach combines two penalty functions. The first derivative of the Canny operator is employed as a roughness penalty function to improve the high frequency information content of the image and a smoothing penalty term is used to remove noise. An adaptive algorithm is used to select the roughness and smoothing control parameters. We evaluate our approach using the Richardson–Lucy EM algorithm as a benchmark. The results highlight some of the difficulties in restoring blurred images that are subject to noise and show that in this case an algorithm that uses a combined penalty function is able to produce better quality results.