During the Oblivion Crisis, gates to the plane of Oblivion opened all over Morrowind. The Empire pulled most of its forces out of Morrowind to deal with the gates in Cyrodiil, leaving the Dunmer people virtually defenseless. There was no standing army at the time, just uncoordinated pockets of resistance. House Redoran was the first House to take up arms against the Daedra and closed some of the Oblivion gates ending with the destruction of their capital city, Ald'ruhn. Over several years they built up an army to be reckoned with. House Telvanni followed suit, and used its long history in magic and the occult to study and, eventually, begin closing the gates of Oblivion.
The Empire lost virtually all its power over Morrowind, and never regained full control of the province. House Hlaalu was affected most by the actions of the Empire, as they welcomed Imperial law and customs with open arms, and benefited from it by gaining unrivaled political and economic strength.
Having past ties to the Empire immediately turned House Hlaalu into the scapegoat for the Dunmer people's suffering. This led to House Hlaalu no longer being recognized as a Great House and were dismissed from the Council. In the Fourth Era, House Sadras took their place as a Great House of Morrowind.