From the point on when the fight against him started, he teleported around like a maniac, like 2 times a second. Every few seconds he would also say stuff like "What was that?" or "Keep your distance... That spell looks dangerous", and the combat music would restart, suggesting that the fight would somehow always restart itself. After I finally defeated him and activated the elevator after running after his teleportations for about 10 mins, the game crashed to desktop. Reloaded a save before, did the fight again (still buggy), ctd at the same moment. Now I probably have to do the whole dungeon again.
(I use mods, but it is highly unlikely that these are the problem since they don't interfere with anything that could be connected to this quest)