That's the entrance to the Theives Guild, I'm guessing you haven't done that questline yet.
What do you think?