Short of murder, there are several places to put your stuff.
*As mentioned above, if you progress along a Great House faction questline, you will eventually get to build a manor. You can choose from Hlaalu, Redoran or Telvanni, but only one.
*Once you join a guild, you can stow stuff in the bedroom areas, including baskets, chests and closets. See imgur.com/a/Kr6jI9d for a bedroom area in the Vivec Mages Guild belonging to my current Nerevarine (they actually expanded into the bedroom across the way, but that's another story).
**Same is true of joining the Temple.
*Note that containers have finite capacity roughly corresponding to their size. This is why I don't just throw everything in the closet, though that closet is nearly packed with alchemical ingredients (the enchanting tools are in the other room).
You'll find that placing items ("home decorating") is much easier in Morrowind than in Skyrim and things don't move around when you aren't looking. ;)