[Pansy-Mae]

After I've thought more about this, I'm thinking these are two possibilities to explain the skinny comment.

1. He assumes I have such a skinny stomach/waist that it creates projection in the breast area.
   
2. He assumes I was overweight at one time. However, I lost a lot of the weight but somehow for one reason or another have kept the breasts.
   

I think there is some truth to the first possibility. After all, I've been achieving a feminine fat redistribution. The second possibility is false as I've never been overweight.

Most guys with 'moobs' are overweight to greater or lesser extent and with a shirt on you can generally see a stomach bulge as well as chest swelling and nipples. If you are anything like me you shirt sticks out at your nipples and then hangs straight down, possibly with a valley in the centre.
I suspect that's what he picked up on, probably at an almost subconcious level?

I realize it may seem like I'm making a big deal out of nothing, but I feel like this is some sort of rite of passage.

I don't think you are making an unnecessarily big deal out of it and I can sympathise with the rite of passage feeling. It is somehow weirdly reassuring to know that they must be real and not a figment of your own imagination, because somebody has noticed them, even if that recognition is itself not very welcome!
I remember the first time it happened to me, I was at an Egyptian airport. I came through the scanner and turned to collect my bags, which stretched my shirt across my boobs and I saw a security guard clock my chest. He came straight over and frisked me to check if I had anything concealed under my shirt. I was soooo glad I wasn't wearing a bra, but unwelcome as it was, I felt strangely elated as well!