PC won't turn on when GPU & HDD plugged in

Just had my 6y.o PC died suddenly, like totally dead and was advised to upgrade the motherboard and processor. So I did and now it only boot successfully if I plugged off the GPU & HDD, if either of them was plugged in the PC just wouldn't turn on at all, like nothing happening in the board, no fan spinning, no light turning, nothing. I tried putting older GPU in and it worked but I still can't plug in my HDD or else it's back to not turning on state. Also tried putting the HDD without any GPU and it still didn't work.

What might be the problem?

Current specs : AMD Ryzen 5 5600 Gigabyte B550M DS3H Corsaire Vengeance LPX 2666MHz 2x8gb MSi Armor MK2 Radeon RX 570 (the one causing issue) Asus Strix GTX 960 (the old one that "fixed" the issue) Corsaire CX750M XPG 1 TB PCIe 3 WD Blue 1 TB PCIe 4 WD Blue 1 TB HDD (the one causing issues)

Pls help I'm not really computer building savvy.