Yeah, it's a well known problem. Mainly it's a balancing issue, which, as with most balancing issues won't really be dealt with until the final Balance and polish passes shortly before final release. Also, the devs have stated that they want to add ideology choices to things like some exploration, invasion and galaxy events so that they aren't just dependent on colony choices and structures. It's a part of the game that is in and working, but not complete.
