I once heard that an abandoned tower used to stand outside Daggerfall, but I never imagined that it still existed. The Red Rooks have been seen searching the area, which leads me to believe that something is there. If it is a fort, think of the treasures that might still be hidden inside!
If you decide to check it out, though, be very careful. The Red Rooks always send their most dangerous members on those sorts of exploration missions.