User blog comment:Zippertrain85/Dawnguard Follow up./@comment-17114085-20131216013719/@comment-17114085-20131216020557

Harald was the First High KIng, he is the 13th in Ysgramor's line so for a long time Skyrim was divided, also the Dragon Priests ruled until the Dragon Wars, which was after Ysgramor came to Skyrim, after the Snow Elves were wiped out. It is entirely possible after the the Dragon Wars the descendants of Ysgramor ruled Eastmarch(since Ulfric is descendat of Ysgramor ad the other Jarls aren't) and other nobles ruled the other Holds. Harkon could have been a noble who conquered part of Skyrim and became a King, just not a High King