Here is a simple idea that be kind of nice. When the village levels up to level 2, the town center tile should upgrade to something in line we the specialization choice. So :
Fortress : the tile should represnt a keep. When at level 3, a castle, etc. up to level 5 with a full flege citadel.
Conclave : the center town could become a chapel, or other type of temple up to a cathedral.
Cities : There are ok as they are.
The castles and the temples should be high enough buildings so they are visible easily. Imagine a castle with all the buildings around ...
I love this game, it's so fun to see it evolve with each new version !