There is one of two things happening:
1) Your planets are being re-invaded by enemy transports. If this is the case, the easiest solution is to park a ship in orbit of the planet. At this point in the game the AI will not attack a defending fleet no matter how weak, so you only have to park a single ship at each planet.
2) Your conquered planets are getting culture-flipped back to their original owners. If this is happening, the best solution is to invade high-influence planets first, so they don't have the influence in the area to flip them back. Note that if you have any ships in orbit when a planet is culture flipped, those ships will flip as well and you will have to take them out in order to re-take the planet.
Unfortunately, the game does not currently have alerts for either of these events, so if you aren't specifically watching the planet at the time, it can be tough to know which it is.
