Board Thread:Skyrim/@comment-124.182.212.163-20140321222851/@comment-4057437-20140411043251

Yes, the Dragonborn's skills go down, and none of the perks will be in effect, either, until they are reassigned.

Any increases in the warhammer's effectiveness due to smithing and enchanted apparel would remain, though, as would the effects of potions (until they wear off).