It may have been for the better. I'm not sure the Nords are capable of dealing with the intricacies of governing an Empire. Alongside their native aggression, their fierce independence leads me to believe that they couldn't govern efficiently. Just look at Skyrim; its divided into nine independent holds that operate freely of one another, which isn't a very efficient means of governance. This isn't me being prejudice; I simply believe some races can lead while others cannot. I wouldn't trust my favorite race, the Khajiit, with leading an Empire.
That sounds kinda racist...
Maybe it is, but its how I feel. Some races make better leaders than others.