There is a growing need to understand the global impact of poverty on early brain and behavioural development, particularly with regard to key cognitive processes that emerge in early development. Although the impact of adversity on brain development can trap children in an intergenerational cycle of poverty, the massive potential for brain plasticity is also a source of hope: reliable, accessible, culturally-agnostic methods to assess early brain development in low resource settings might be used to measure the impact of early adversity, identify infants for timely intervention, and guide the development and monitor the effectiveness of early interventions. Visual working memory (VWM) is an early marker of cognitive capacity that has been assessed reliably in early infancy and is predictive of later academic achievement in Western countries. Here, we localized the functional brain networks that underlie VWM in early development in rural India using a portable neuroimaging system, and we assessed the impact of adversity on these brain networks. We recorded functional brain activity as young children aged 18-207 weeks performed a visual working memory task. Brain imaging results revealed localized activation in the frontal cortex, replicating findings from a midwestern US sample. Critically, children from families with low maternal education and income showed weaker brain activity and poorer distractor suppression in canonical working memory areas in the left frontal cortex. Implications of this work are far-reaching: it is now cost-effective to localize functional brain networks in early development in low-resource settings, paving the way for novel intervention and assessment methods.