A large fragment of the Ehlnofey world landed on Nirn relatively intact, and the Ehlnofey living there were the ancestors of the Mer. These Ehlnofey fortified their borders from the chaos outside, hid their pocket of calm, and attempted to live on as before. Other Ehlnofey arrived on Nirn scattered amid the confused jumble of the shattered worlds, wandering and finding each other over the years.
Eventually, the wandering Ehlnofey found the hidden land of Old Ehlnofey, and were amazed and joyful to find their kin living amid the splendor of ages past. The wandering Ehlnofey expected to be welcomed into the peaceful realm, but the Old Ehlnofey looked on them as degenerates, fallen from their former glory. For whatever reason, war broke out, and raged across the whole of Nirn.
The Old Ehlnofey retained their ancient power and knowledge, but the Wanderers were more numerous, and toughened by their long struggle to survive on Nirn. This war reshaped the face of Nirn, sinking much of the land beneath new oceans, and leaving the lands as we know them (Tamriel, Akavir, Pyandonea, Atmora, and Yokuda). The Old Ehlnofey realm, although ruined, became Tamriel. The remnants of the Wanderers were left divided on the other three continents.
The Wandering Ehlnofey are the ancestors of Men and the Old Ehlnofey are the ancestors of Mer, according to The Annotated Anuad.