Board Thread:General Discussion/@comment-75.137.209.142-20140823194421/@comment-4544033-20141002201742

Their souls where not trapped in a soul gem and sent to Oblivion, and they MIGHT be-able to exist through the gate, but they are connected to the Cairn, so I doubt they could stay away long.

On top of that, when you speak to most of the spirits, they lost hope, they are depressed and have given up. The only ones to leave the realm are that horse and Durnehviir, and they can't stay away forever, they have to return.