Board Thread:Lore Discussion/@comment-25528618-20141016215758/@comment-25569410-20141022205148

Association with the Thalmor is probably the strongest reason.

They are slightly smug and pretentious, of course this depends on which Altmer you are talking to. They are obviously very highly gifted in the Arcane arts and this is a constant source of their "superiority" throughout the ages!

In Skyrim everything is accentuated as the Thalmor are basically the cause of the Civil War and are also rounding up Talos worshippers wherever they find them.

There are many example of this IRL also, just look at countries which have been under foreign rule and gain independence (generations later there is still resentment between both parties).