Note that most horses you encounter are likely to be owned by someone, even if you find the horse in the middle of the wilderness. It is still considered a stolen horse, so dismounting and then mounting the horse in plain sight will incur a bounty. However, stealing a horse while hidden in sneak mode allows you to ride the horse without incurring a bounty. Stealing a horse incurs a 50 gold bounty in that hold which stacks every time a guard notices (or is told about) you mounting a stolen horse (even if you dismount and then mount the same stolen horse). To steal a horse, simply mount it and start riding. Stealing horses is considered to be a crime. The only two exceptions to this are Shadowmere and Frost. All horses have the same speed, stamina, and health, varying only in color. Horses can be purchased from stables, located outside each major city.