Background: We investigated the associations between vitamin D status in early and late pregnancy with neonatal small for gestational age (SGA), low birth weight (LBW) and preterm delivery. Furthermore, associations between vitamin D status and pregnancy loss were studied. Methods: Serum 25-hydroxyvitamin D (25OHD) was sampled in gestational week ≤ 16 (trimester 1 (T1), N = 2046) and > 31 (trimester 3 (T3), N = 1816) and analysed using liquid chromatography tandem mass spectrometry. Pregnant women were recruited at antenatal clinics in south-west Sweden at latitude 57–58°N. Gestational and neonatal data were retrieved from medical records. Multiple gestations and terminated pregnancies were excluded from the analyses. SGA was defined as weight and/or length at birth < 2 SD of the population mean and LBW as < 2500 g. Preterm delivery was defined as delivery < 37 + 0 gestational weeks and pregnancy loss as spontaneous abortion or intrauterine fetal death. Associations between neonatal outcomes and 25OHD at T1, T3 and change in 25OHD (T3-T1) were studied using logistic regression. Results: T1 25OHD was negatively associated with pregnancy loss and 1 nmol/L increase in 25OHD was associated with 1% lower odds of pregnancy loss (OR 0.99, p = 0.046). T3 25OHD ≥ 100 nmol/L (equal to 40 ng/ml) was associated with lower odds of SGA (OR 0.3, p = 0.031) and LBW (OR 0.2, p = 0.046), compared to vitamin D deficiency (25OHD < 30 nmol/L, or 12 ng/ml). Women with a ≥ 30 nmol/L increment in 25OHD from T1 to T3 had the lowest odds of SGA, LBW and preterm delivery. Conclusions: Vitamin D deficiency in late pregnancy was associated with higher odds of SGA and LBW. Lower 25OHD in early pregnancy was only associated with pregnancy loss. Vitamin D status trajectory from early to late pregnancy was inversely associated with SGA, LBW and preterm delivery with the lowest odds among women with the highest increment in 25OHD. Thus, both higher vitamin D status in late pregnancy and gestational vitamin D status trajectory can be suspected to play a role in healthy pregnancy.