Some interesting ideas here. One question though. What would incentivize you to be the first to build big ships then? ie what would be the advantage of committing to the research, then the industry to churn out bigger ships, when the smaller ships you can already build are better at fighting the existing smaller ships really?
Or are you envisioning that you'd need ships with more "penetration" to push down starbases?
There would be just as much incentive to build bigger ships as currently exists. You get far more hit points, and so are more likely to survive the battle. Because you only go up one step at a time (from tiny to small, from small to medium, etc.), and because the impact of maneuverability/resilience is negligible between ships that are one size apart, it will be advantageous be one level ahead of your opponent, just like it is now.
Actually, it's not true that the smaller ships would be better at fighting other small ships. They would be better at doing damage to them. They would also be taking more damage. The way it works is small ships apply more damage to small ships and big ships apply more damage to big ships.
I conceive of the damage avoided by small ships being expressed in terms of evasion and missed shots. I conceive of the damage avoided by large ships as being expressed in terms of damage reduction or resistance. Mathematically it would come out to an even playing field on average.
For most battles, all this would really mean is that they would take a bit longer. For some battles, that would mean the turn limit is reached, and the battle ends. This possibly lets one side escape.
I don't remember what the turn limit is, but let's say it is 50. (If this system were adopted, the turn limit would probably have to be changed anyway.) Let's say you have a fleet of tiny hulls versus a fleet of massive hulls. Let's say the massive hull fleet can only fit in 2 ships within the logistical limit. And let's say with the drastic difference in maneuverability, the massive hull ships miss 99% of their shots. With 50 shots (because there are 50 turns) that means that each massive hull ship has a 50% chance this battle to hit (and kill) a tiny hull ship this combat. Since there are two of them, that means that on average they will kill one little ship each time these fleets engage in combat. Since the tiny ships are doing drastically reduced damage and the big ships are healing every turn, the big ships seem to have a slight edge here.
To take another example, let's say you have one massive hull fighting one tiny hull. Following the numbers laid out in the previous example, there would be a 50% chance the tiny ship is destroyed, and a 50% chance that it manages to "get away." This seems to approximate what "should" happen. If this were "real life," the tiny hull would be warming up the hyper drive to get away, not recklessly trying to destroy the much larger ship.
Of course, going after a whole fleet of tiny ships with your massive ships would just be silly. That's why you send your small and medium ships after them.
I wasn't really thinking about this combat system as it relates to starbases, but I suppose that thinking of starbases as another kind of massive hull is one possibility.
There could additionally be one decisive advantage of larger hulls that would make them essential. That is the concept of taking and giving ground (or I guess space). Currently, if a battle ends at the turn limit both sides remain in the hexes they fought from. Battles are fought between hexes and not on them. I think what should happen is the side with more hit point at the end of the fight should control the hex the fight took place on. Let's say the attacking force wins. "Winning" here just means they had more total hit points at the end of the fight, regardless of whatever casualties they incurred. The attacking force would then take over the hex previously occupied by the defending force, pushing the defending force back one tile. This would give fleets with more hit points (i.e., bigger ships) an advantage in taking over hexes and defending them. Big ships would therefore play a pivotal role in launching and repelling invasions. An invasion, like a battle, wouldn't occur between two hexes but solely on the hex occupied by the planet. So long as the fleet is not defeated or pushed off the hex, the transports can continue to invade the planet. (This is a change I would like to see regardless of everything else. Transports shouldn't be interrupted in their mission just because the fleet defending them is attacked.)
There are two kinds of warfare in GalCiv, and what the proposed system does is make them more distinct. Tiny hulls excel at inflicting economic damage, or screening against enemy "pirates." (Tiny hulls already innately have more moves per turn; I would not change that.) Massive hulls excel at defending or attacking key positions. Medium hulls are multi-purpose; they can reliably take part in both kinds of warfare without excelling in either. Carriers would have the advantages of both large and small ships, making them the best all-around, just like they are now.