Why Weren’t PJ More Commercially Successful In The 2000s?

Curious to hear what y’all think. Certainly Binaural and Riot Act weren’t going for as much of a commercial sound so I get that, but Avocado and Backspacer sound to me more like mainstream targeted albums. From that time period I think about albums like American Idiot and Stadium Arcadium, which were wildly successful. I think PJ are overall bigger than both Green Day and RHCP, so why did they get beat out during that time period?