I agree with you that many of life's best adventures are spontaneous and serendipitous. However, in your wanderings, a handful of places you might want to see if you find yourself in the neighborhood:
Green Apple Books & Music on Clement St. is a small shrine to the cult of bibliomania. Clement in general is a fun place, but Green Apple is my favorite.
I don't know if you'll be there when all the trees are in bloom, but even if you're not, there's something very special about the Palace of Fine Arts. Also, if you want to release the inner kid, a visit to the Exploratorium is always fun.
There are numerous beaches, all of which have their followers. I used to frequent Stinson Beach for swimming and big ol' Ocean Beach to scrounge for sand dollars.
All sorts of interesting sights are to be had in Golden Gate Park. I'm not sure whether Steinhart Aquarium is back up and running (it was closed for renovation the last time I was there), but it's fun. The Japanese garden in the same general area is lovely--good for some quiet, meditative strolling.
Walking or biking across the Golden Gate Bridge should probably be on that list of things to do before you die.

If you're a live theater fan (and who isn't?), you should see what's playing at the ACT (American Conservatory Theater) while you're there.
Buy a cheap kite and let fly!
Hope something here proves to be of use. San Francisco is a great place; I think you'll love it.