The maintenance of genetic variance in fitness represents one of the most longstanding enigmas in evolutionary biology. Sexually antagonistic (SA) selection may contribute substantially to maintaining genetic variance in fitness by maintaining alternative alleles with opposite fitness effects in the two sexes. This is especially likely if such SA loci exhibit sex-specific dominance reversal (SSDR)—wherein the allele that benefits a given sex is also dominant in that sex—which would generate balancing selection and maintain stable SA polymorphisms for fitness. However, direct empirical tests of SSDR for fitness are currently lacking. Here, we performed a full diallel cross among isogenic strains derived from a natural population of the seed beetle Callosobruchus maculatus that is known to exhibit SA genetic variance in fitness. We measured sex-specific competitive lifetime reproductive success (i.e., fitness) in >500 sex-by-genotype F 1 combinations and found that segregating genetic variation in fitness exhibited pronounced contributions from dominance variance and sex-specific dominance variance. A closer inspection of the nature of dominance variance revealed that the fixed allelic variation captured within each strain tended to be dominant in one sex but recessive in the other, revealing genome-wide SSDR for SA polymorphisms underlying fitness. Our findings suggest that SA balancing selection could play an underappreciated role in maintaining fitness variance in natural populations.