My brother called me gay and we stopped speaking for a while

My brother called me the f slur randomly when we got into a disagreement along with other homosexual slurs and I hated him for it, I knocked the shit out of him for it too. I was 13-14 and he’s three years younger than me. Although I am gay, it truly bothered me because I come from a VERY conservative religious household, the mere mention of homosexuality makes my family mad. I didn’t want them to get onto me before I had any independance, it truly boiled my blood, how could he do that? I stopped speaking to him for years even tho we live under the same roof. I was also mad everyone took my brothers side including my sister and mom and shamed me for beating him up just cuz I was older. Mind you, not the first time they did it so I already had some resentment, everytime we got into fights they’d take his side. Reconciled the relationship eventually but I still feel weird about the whole situation, I forgive him ofc but I perhaps should’ve earlier. Yes I hold grudges, he’s straight and I have been in survival mode in my household since the time I can remember, I definitely felt like he compromised that at the time.