Its because of the quest, you have to go.
The mines are good. You get shivs, fist fight a big orc, and they touch your no-no spot O.o
Yep it's part of the quest, if you don't want to go to jail don't start the quest (and fighting it can cause bugs). I don't really like it either, i did it once or twice but ignore it on most playthoughs.
I love jail break quests... :/ changes things up... There IS a way to avoid it, though... you have to snipe the guards via arrows... before interacting with them as they stand over Eltry's body... or at least, supposedly that works...
Yes, I know this is a month plus old thread, but maybe someone else will see it.
I once glitched out of the quest by having a few hostile NPCs enter the shrine with me and tabbing out of the dialog with a guard (so that I don't get killed). After everything settled, I just paid off my bounty and was set free.