I think this depends on the woman. I would say probably most women wouldn't necessarily find that a turn on, but I'm sure some would. I believe there is someone out there for everyone. And even if there is a woman who isn't outright turned on by it, she may not be turned off by it either. There are a lot of very open minded people out there, but many times it doesn't feel like it when we're looking for it.
Everyone has a different fetish or kink they like, or some don't like any and prefer a strict diet of blandness. But, if that's what they enjoy, then more power to them.
I would suggest putting yourself out there online and seeing if you can catch a woman who enjoys seeing her man in her clothes. If it's OK for a girl to wear guy's clothes why not the other way around?
I don't think it should be too hard to find someone out there for you.
Adult Friend Finder has a button for transsexual/transvestite/transgendered... Even though you do not go out in drag I would use this so you can be sure to find women who find that attractive.
I will be interested to see if Dan and Jennifer have anything to say on this. I'll also be interested to hear how you make out with this. I wish you the best of luck, and remember.. don't let the bastards grind you down..