Here is a simple idea that be kind of nice. When the village levels up to level 2, the town center tile should upgrade to something in line we the specialization choice. So :

Fortress : the tile should represnt a keep. When at level 3, a castle, etc. up to level 5 with a full flege citadel.

Conclave : the center town could become a chapel, or other type of temple up to a cathedral.

Cities : There are ok as they are.

The castles and the temples should be high enough buildings so they are visible easily. Imagine a castle with all the buildings around ...

I love this game, it's so fun to see it evolve with each new version !