Phytoplankton concentrations display strong temporal variability at different time scales. Recent advances in automated moorings enable detailed investigation of this variability. In this study, we analyzed phytoplankton fluctuations at four automated mooring stations in the North Sea, which measured phytoplankton abundance (chlorophyll) and several environmental variables at a temporal resolution of 12–30 min for two to nine years. The stations differed in tidal range, water depth and freshwater influence. This allowed comparison of the predictability and environmental drivers of phytoplankton variability across different time scales and geographical regions. We analyzed the time series using wavelet analysis, cross correlations and generalized additive models to quantify the response of chlorophyll fluorescence to various environmental variables (tidal and meteorological variables, salinity, suspended particulate matter, nitrate and sea surface temperature). Hour-to-hour and day-to-day fluctuations in chlorophyll fluorescence were substantial, and mainly driven by sinking and vertical mixing of phytoplankton cells, horizontal transport of different water masses, and non-photochemical quenching of the fluorescence signal. At the macro-tidal stations, these short-term phytoplankton fluctuations were strongly driven by the tides. Along the Dutch coast, variation in salinity associated with the freshwater influence of the river Rhine played an important role, while in the central North Sea variation in weather conditions was a major determinant of phytoplankton variability. At time scales of weeks to months, solar irradiance, nutrient conditions and thermal stratification were the dominant drivers of changes in chlorophyll concentrations. These results show that the dominant drivers of phytoplankton fluctuations differ across marine environments and time scales. Moreover, our findings show that phytoplankton variability on hourly to daily time scales should not be dismissed as environmental noise, but is related to vertical and horizontal particle transport driven by winds and tides. Quantification of these transport processes contributes to an improved predictability of marine phytoplankton concentrations.