No, don't remove minor (independent) single planet civilizations! This tradition goes back decades. It would make the game less complex and less awesome!
I just played a 2 player tiny map with six systems. Two were player systems, two were colonizable, and two required invasion fleet of transports. This was a perfect game. Lots of choices and rewards for making good choices.