Reintroducing breast milk

My 4 month old was diagnosed with CMPA at around 1.5 months. Since then, I’ve stopped breastfeeding and started using hypoallergenic formula. He was reacting to more than just dairy and I really couldn’t restrict my diet even more. His pediatrician told us to try some of my frozen breast milk and see if he still reacts to it. All of the resources online say to reintroduce at 9-12 months, not 4 months. Should I wait or follow my pediatrician advice??