Scenarios that limit global warming to 1.5 °C describe major transformations in energy supply and ever-rising energy demand. Here, we provide a contrasting perspective by developing a narrative of future change based on observable trends that results in low energy demand. We describe and quantify changes in activity levels and energy intensity in the global North and global South for all major energy services. We project that global final energy demand by 2050 reduces to 245 EJ, around 40% lower than today, despite rises in population, income and activity. Using an integrated assessment modelling framework, we show how changes in the quantity and type of energy services drive structural change in intermediate and upstream supply sectors (energy and land use). Down-sizing the global energy system dramatically improves the feasibility of a low-carbon supply-side transformation. Our scenario meets the 1.5 °C climate target as well as many sustainable development goals, without relying on negative emission technologies.