Do chromosomes really make a man?

I follow a good number of atheist and skeptic oriented groups on social media (I consider myself an agnostic atheist). For the most part, I've come to expect a level of critical thought and maturity out of these groups. Ever since Caitlyn Jenner made her debut on the cover of Vanity Fair, the topic of her transition has popped up in a few of these communities. How this topic was relevant to those groups, I couldn't say. But it was there and for the most part the comments were positive and supportive. But I was surprised by the handful that continued to make the small minded or uninformed comments on the subject we've come to expect from more conservative parties. There was the classic, "Why would you cut off a perfectly good penis" line of thought and also a frequent sentiment of "I'll call Jenner 'her' when I see XX chromosomes."
There is a widespread attitude that XY chromosomes are an indelible indicator of maleness. Clearly that is not the case, as Androgen Insensitivity Syndrome shows us. Individuals born with complete Androgen Insensitivity Syndrome appear to be outwardly female, are typically raised as girls, and identify as women in all psychological and social facets. Often, they are unaware of the condition until reaching their teenage years, when their lack of menstrual cycle provides the first hints that something is amiss. These women have XY sex chromosomes, but because of a mutation in the Androgen Receptor gene, their bodies never masculinize and continue developing based on the female model.
The existence of AIS should be enough to show that physical sex and gender identity are not fully dependent upon our chromosomes. Nature simply doesn't work in such a cut & dry manner. But of course, AIS is one thing. Being transgender is something altogether different. Or is it?
A study conducted at Prince Henry's Institute of Medical Research in Australia found a difference in the Androgen Receptor gene among male-to-female transgender participants. This is noteworthy, because it is a mutation in this same gene that results AIS. While the Australian study has a relatively small sample size (112 transgender participants) and so cannot be considered conclusive, it does raise intriguing questions. I can't help but speculate on the possibility that perhaps gender dysphoria results from a condition similar to AIS, lacking the overt physical indications but preventing full masculinization of the brain. Earlier studies have already found evidence that the brain structures of transgender individual more closely match those of the gender with which they identify. These studies controlled for the effects of transgender hormone replacement therapies. Considering that AIS is already observed in a wide spectrum of degrees from Mild (MAIS) to Complete (CAIS), my speculation isn't completely groundless.
While the studies are intriguing, there is need for replication in order to confirm the findings. Gender dysphoria is a very under-researched topic, so answers will be slow in coming. But it is my suspicion that in time it will come to be considered another of the intersex conditions. It is doubtful that this will do much to change society's more small-minded attitudes toward transgender individuals. But it would at least affirm our feelings of a disconnect between our physical bodies and our feelings of identity. We aren't delusional. There is a literal conflict between what our brains are telling us, and our physical reality.