That is not a bad CPU, it might bottle neck things a bit for 2x1080s but losing 10-20 FPS in mGPU mode seems a lot, but who knows, with 2x1080s a huge amount of stuff going on.
How is your RAM configured? It seems to make a really big difference in this game. Do you have 2 sticks but 4 RAM slots? If I am remembering correctly to take advantage of dual channel in that kind of configuration the cards should not be side by side, but in slots 1 and 3 or 2 and 4. Check with your MB manual. If you have 2 sticks and 2 RAM slots then you can ignore me. 
(here is a link to the Steam forums with a couple of 1080 owners talking about RAM and performance, perhaps the DDR4 RAMs higher bandwidth is a big help)
http://steamcommunity.com/app/228880/discussions/1/358415206091503844/
There was another thread where a 1080 user or 2x1080 user overclocked their RAM too and got a significant improvement but I can't find it. So overclock RAM and CPU?
Have you tried testing at 1080p instead of 4K? Normally the lower the resolution the greater the chances of a CPU bottleneck while with a high resolution the greater chances of a GPU bottleneck. If you try this at 1080p or 1440p is the FPS reduction even worse relative to 1 card vs 2?
