Fishing has wide-ranging impacts on marine ecosystems. One of the most pervasive signs of intensive fishing is "fishing down the food web", with landings increasingly dominated by smaller species from lower trophic levels. Decreases in the trophic level of landings are assumed to reflect those in fish communities, because size-selective mortality causes decreases in the relative abundance of larger species and in mean body size within species. However, existing analyses of fishing impacts on the trophic level of fish communities have focused on the role of changes in species composition rather than size composition. This will provide a biased assessment of the magnitude of fishing impacts, because fishes feed at different trophic levels as they grow. Here, we combine body size versus trophic level relationships for North Sea fishes (trophic level assessed using nitrogen stable-isotope analysis) with species-size-abundance data from two time-series of trawl-survey data (whole North Sea 1982-2000, central and northern North Sea 1925-1996) to predict long-term trends in the trophic structure of the North Sea fish community. Analyses of the 1982-2000 time-series showed that there was a slow but progressive decline in the trophic level of the demersal community, while there was no trend in the trophic level of the combined pelagic and demersal community. Analyses of the longer time-series suggested that there was no trend in the trophic level of the demersal community. We related temporal changes in trophic level to temporal changes in the slopes of normalised biomass size-spectra (which theoretically represent the trophic structure of the community), mean log2 body mass and mean log2 maximum body mass. While the size-based metrics of community structure showed long-term trends that were consistent with the effects of increased fishery exploitation, these trends were only correlated with trophic level for the demersal community. Our analysis suggests that the effects of fishing on the trophic structure of fish communities can be much more complex than previously assumed. This is a consequence of sampled communities not reflecting all the pathways of energy transfer in a marine ecosystem and of the absence of historical data on temporal and spatial changes in the trophic level of individuals. For the North Sea fish community, changes in size structure due to the differential effects of fishing on species and populations with different life histories are a stronger and more universal indicator of fishing effects than changes in mean trophic level.