Estimated heritability of educational attainment (EA) varies widely, from 23% to 80%, with growing evidence suggesting the degree to which genetic variation contributes to individual differences in EA is highly dependent upon situational factors. We aimed to decompose EA into influences attributable to genetic propensity and to environmental context and their interplay, while considering influences of rearing household economic status (HES) and sex. We use the Project Talent Twin and Sibling Study, drawn from the population-representative cohort of high school students assessed in 1960 and followed through 2014, to ages 68−72. Data from 3552 twins and siblings from 1741 families were analyzed using multilevel regression and multiple group structural equation models. Individuals from less-advantaged backgrounds had lower EA and less variation. Genetic variance accounted for 51% of the total variance, but within women and men, 40% and 58% of the total variance respectively. Men had stable genetic variance on EA across all HES strata, whereas high HES women showed the same level of genetic influence as men, and lower HES women had constrained genetic influence on EA. Unexpectedly, middle HES women showed the largest constraints in genetic influence on EA. Shared family environment appears to make an outsized contribution to greater variability for women in this middle stratum and whether they pursue more EA. Implications are that without considering early life opportunity, genetic studies on education may mischaracterize sex differences because education reflects different degrees of genetic and environmental influences for women and men.