I may have found a contributing factor. When I go to the "Opponent Settings" screen, it starts at 15 opponents so I just accepted that default. However, as the second screen snapshot below shows, when you hover over the top sections that says "15/15 Max Players" a little popup appears that shows that the recommended number of opponents is 8 for the "A Few" Number of Sectors galaxy setting.
Wouldn't it make more sense for the default number of opponents to be the recommended number of opponents for the selected Number of Sectors? Also, why wouldn't you make the recommended number of opponents more obvious? Why would you make the player find that information by accidentally hovering over the top box?

