Guided trail rides are around $55 for an hour or $75 for two hours. Roosevelt Lodge, which also offers stagecoach tours, offers rides in an especially beautiful area of the park, with rolling hills and waving grasses.
Like a lot of the non-boating and non-fishing crowd, I’d assumed swimming in Yellowstone was prohibited. But while it’s not really advertised, you can legally swim just about any place there’s enough water to do it (the exceptions being the hot springs, where you’ll definitely want to avoid the boiling pits of sulphur).
You don’t see many people swimming in Yellowstone partly because people don’t know they have the option, but mainly because the water is extremely cold, even in summer. The picnic area at Lava Creek Trail is a (relatively) popular spot for taking a cold dip, but there’s also a handful of naturally warm thermal sites -- try Boiling River (north of Mammoth) and Firehole Canyon Drive (down by Madison). You’re safe from any unpleasant things like leeches or snapping turtles, although there is technically a brain-eating amoeba that you could contract in the warm water. Don’t worry though, there have never been any reported cases of swimmers contracting it -- park rangers just advise not dunking your head.