User blog comment:LynCorbray/Winterhold/@comment-10207438-20130713235914

1a. Just because they're a big city doesn't mean they need walls.

1b. also back then they had great relations with the college of winterhold, it's possible that back then they had a magic wall.

2. there are some piles of wood and even a couple skeletons down under the cliff, you gotta look for it though, you're not gonna see it from the top of the cliff.

3. the houses that were at the edge probably slipped off the foundations and fell into the waves down below.

You're not "missing" anything, you're just expecting a bit too much.