I'm also a small-ish frame male, and at this point I've had the opportunity to (in some cases, attempt to) carry a large variety of handguns concealed; everything from a Glock 20 to a Ruger SR40c to a Bersa Thunder CC to a Beretta 92...
I highly recommend the Sig Sauer P938. Light, slim, metal frame; handles and shoots like a larger handgun, but disappears in my CrossBreed Mini-Tuck.
If the pseudo-1911 battery-of-arms isn't a concern for you, find one for rent at a range and give it a try.
Alternatively, with the right modern ammo a .380 is nothing to sneeze at. I have a Ruger LCP II which I pocket carry when even the Sig is too heavy/large to conceal comfortably (ie. when exercising). I also highly recommend the LCP II. If budget is a concern, this is a great place to start.
I usually carry an extra magazine for each of those, in a SneakyPeteHolsters slim leather mag pouch.
Bottom line, though - find a range that'll let you rent a few and see how they feel. Different folks like different kinds of triggers, build materials, batteries-of-arms, etc.