Intralocus sexual conflict (IaSC) occurs when selection at a given locus favors different alleles in males and females, placing a fundamental constraint on adaptation. However, the relative impact of IaSC on adaptation may become reduced in stressful environments that expose conditionally deleterious mutations to selection. The genetic correlation for fitness between males and females (rMF) provides a quantification of IaSC across the genome. We compared IaSC at a benign (29°C) and a stressful (36°C) temperature by estimating rMFs in two natural populations of the seed beetle Callosobruchus maculatus using isofemale lines. In one population, we found substantial IaSC under benign conditions signified by a negative rMF (−0.51) and, as predicted, a significant reduction of IaSC under stress signified by a reversed and positive rMF (0.21). The other population displayed low IaSC at both temperatures (rMF: 0.38; 0.40). In both populations, isofemale lines harboring alleles beneficial to males but detrimental to females at benign conditions tended to show overall low fitness under stress. These results offer support for low IaSC under stress and suggest that environmentally sensitive and conditionally deleterious alleles that are sexually selected in males mediate changes in IaSC. We discuss implications for adaptive evolution in sexually reproducing populations.