Grytviken is the whaling station made famous by Shackleton's reunion with civilization on South Georgia after losing his ship, the Endurance, to Antarctic pack ice in 1915. Toast "Bring 'em Back Shack" at his gravesite in a small cemetery overlooking the bay.
The abandoned whaling station has undergone a project to remove all asbestos and dangerous collapsing buildings and may be explored. The station was active in the whaling industry until the middle of the last century. In the meantime the museum, operated by Tim & Pauline Carr, is an excellent place to learn about the natural and whaling history of the island. Other popular visitor destinations include the restored Norwegian church, and the cemetery, which contains the grave of the famous explorer Ernest Shackleton.
The British Antarctic Survey  maintains a presence on the island in a recently renovated research station on nearby King Edward Point.
Hike up to the reservoir. Located on the hill above the whaling station is a fairly large lake. Footing may be muddy, but the scenery is nice and the view of the bay is excellent. Be aware that terns may begin attacking from the air; if they do it means that you are near their nest and should backtrack until they feel you are a safe distance away.
Visit Shackleton's grave. The cemetery is located on the opposite side of the whaling station from the museum. Shackleton is buried here along with many of the whalers who died on South Georgia.
Visit Shackleton's cross. The cross is located at the end of King Edward Point, beyond the British Base. A trail to the cross is fairly easy to follow. While visiting be on the lookout for the very territorial fur seals; should one charge you it is best to carry a long stick or tripod with which to tickle its whiskers, which surprisingly deters most attacks.
Write letters home. There is a mailbox in front of the museum, and stamps and postcards can be purchased in the gift shop. Mail will reach most destinations within two or three weeks. The Post Office is 1km away at King Edward Point, an easy summer stroll, and has a range of postcards, stamps, first day covers, SG coins and a few SG Government publications for sale.
There is a small gift shop in the museum that sells books of local interest, posters, and sundry other souvenirs. British pounds, Falklands pounds, Euros, and American dollars will all be accepted, as will Visa and Mastercard (but not AMEX). The Post Office is open upon request at King Edward Point and may be brought on board larger ships.