Geography, including climatic factors, have long been considered potentially important elements in shaping socio-economic activities, alongside other determinants, such as institutions. Here we demonstrate that geography and climate variables satisfactorily explain the worldwide economic activity as measured by the per capita Gross Cell Product (GCP-PC) at a fine geographical resolution, typically much higher than country average. A 1° by 1° GCPPC dataset has been key for establishing and testing a direct relationship between 'local' geography/climate and GCP-PC. Not only have we tested the geography and climate hypothesis using many possible explanatory variables, importantly we have also predicted and reconstructed GCP-PC worldwide by retaining the most significant predictors. While this study confirms that latitude is the most important predictor for GCP-PC when taken in isolation, the accuracy of the GCP-PC prediction is greatly improved when other factors mainly related to variations in climatic variables, rather than average climatic conditions as typically used, are considered. However, latitude diminishes in importance when only the wealthier parts of the globe are considered. This work points to specific features of the climate system which explain economic activity, such as the variability in air pressure. Implications of these findings range from an improved understanding of why socio-economically better-off societies are geographically placed where they are in the present, past and future to informing where new economic activities could be established in order to yield favourable economic outcomes based on geography and climate conditions.