Feminine or femboy?

So... to start, I'm straight and I'm not in any group, club or anything like that (such as LGBT etc). I'm a male, but with a feminine side. I enjoy art and have always focused on outfits for my character rather than drawing 'normal' things. I was homeschooled so you can imagine the difficulty embracing my femininity, at least around family or my tutor. When I did first show that side of me (online), people got the impression that I was gay, or bi. When I said I'm not, they got mad and this has happened multiple times over the years, so all it did was make it difficult to be open about it. After a while I stopped calling myself a femboy. I consider myself a feminine person personality-wise, but not 100%. It's difficult to explain but think of it as a partial part of who I am, not a style or customary decision (cosmetic, i.e makeup, clothes, long hair etc).

Anyway.. I would draw a lot and loved getting ideas from people who may be interested in sharing their work or giving ideas for mine. It's something I used to constantly. Nowadays it feels like walking on egg shells. Like I've gotten out of touch or haven't been able to stay in touch with how people interact or behave nowadays. I can't say what's on my mind or people misinterpret or outright twist it. I've been called a homophobic person or transphobic just because I am straight. I don't have a problem with people that choose to identify differently than I do, or are attracted to the same gender.

Has anyone else felt this way?