If you haven't heard, Summers basically says that men are overrepresented in the hard sciences because of the long hours required to excel, innate physiological differences, and the lack of self-corrective measures that would indicate an untapped resource base. A lot of pissed off people have countered with accusations of deep systemic biases in education.
I think Summer's may have a point. Feminists have long wanted to treat men and women as if we are exactly the same. We aren't of course. The sexes have physiological differences that start with reproduction but go all the way to brain construction. So the question is how much does this matter? Yglesias points out quite correctly that even if this only matters a little, once you get to the elite levels of scholarship even a small handicap can have large effects.
One of Yglesias commenters had this to say:
In particular, it's known that until the 7th grade, women perform just as well as men on math tests and profess to like mathematics at least as much or more than men do. In between the 7th and 10th grades, however, the percentage of women who claim on surveys to enjoy mathematics and to find it easy plummets, along with the relative scores of women as compared with men.Another commenter replied that this change in performance coincides with puberty. Another replied to him saying that 7th-10th grade is a little late for pubescence to be the major causal factor.
My take is this. During my high school years, 7th to 10th grade is when abstract conceptualization started coming to the fore in my mathematical instruction. Prior to this my mathematics training was in the simple arithmetic of rote study. This was adding, multiplying, order of operations, simple perimeter/area calculations, etc. On the other hand 7th grade was Pre-Algebra (Ms. Gera), 8th Algebra (Mr. Koz), 9th Geometry (Ms. Lonquist), 10th Algebra II/Trig (Mr. Withelder), 11th Pre-Calc (Mr. Roche), 12 AP Calculus (Mr. Owsley). It was when we really started expressing complex problems (for us) in terms of mathematical equations. I was on the advanced/gifted/AP track so figure that others my age could lag up to two or three years behind me.
I seem to recall geometry being an especially tough course that separated the adults from the children. It was almost all abstract geometric conceptualization and proofs with very little in the way of 1+1=2 mathematics. Note that the teacher was a woman so sexism didn't play a part in it.
Amybear may have more to say about this since her thesis is about this sort of thing and I'm just shooting the breeze.
UPDATE: The Volokh Conspiracy is weighing in too.