Akhaltsikhe (Georgian: ახალციხე, "New Fortress") is a small city of about 50,000 and the capital of Samtskhe-Javakheti. The city has been around for at least 800 years, and was a regional administrative center for the Ottomans from the sixteenth century up to the Russo-Turkish War. Until the twentieth century Akhaltsikhe was majority Armenian, but today, unlike most of the province, it is majority Georgian. It's a rather sleepy town, but its Old City is worth a visit, and it's a great base for exploring the surrounding areas, including Sapara Monastery and Khertvisi Fortress in the immediate vicinity.
The principal route to Akhaltsikhe runs from Georgia's main East-West highway (E60) at the spur in Khashuri leading to Borjomi. Marshrutkas run to Akhaltsikhe's main market/bus station from Tbilisi's Didube market, as well as from the bus stations in Kutaisi and Khashuri. Coming from Kutaisi or Tbilisi, it's best to catch your ride early in the morning if you want to avoid finding another marshrutka upon arrival in Khashuri. But it's not terribly difficult to catch a marshrutka going between Akhaltsikhe and Khashuri before dinner time.
Akhaltsikhe is connected to the main Batumi–Tbilisi–Baku railroad by a spur through Borjomi ending in Akhaltsikhe itself. You should be able to find a train going this way once daily from the Batumi, Kutaisi, and Tbilisi train stations.
Sapara Monastery (საფარის მონასტერი) is about 10-12 km outside of Akhaltsikhe up into the mountains. The monastery was established in the tenth century, but the principal church, St. Sabas, was built sometime in the thirteenth century. Until the twentieth century, the monastery had been perfectly preserved, as its hidden location saved it from Ottoman discovery throughout the empire's three-century long control of southwestern Georgia. Alas, the Soviets found it, and abused it in the usual soulless fashion, albeit not to the same extent as many other Georgian Orthodox establishments—the frescoed walls were not whitewashed, and remain in good condition (especially following a recent restoration). During a visit, make sure to climb up the nearby slopes towards a rocky outcropping to get lovely views over the monastery and the valleys in the distance. Also make sure not to use flash photography in the churches, unless you want to see some seriously angry monks. If you can make yourself understood, you can overnight in the monastery's chambers.
The walls of Khertvisi Fortress
Khertvisi Fortress (ხერთვისის ციხე) looms over the village of Khertvisi. The outcrop was used as a fortress from the second century B.C., and was reputedly destroyed by Alexander the Great. The "modern" fortress, however, was built around the thirteenth-fourteenth centuries, and saw fighting during the Ottoman invasion (and subsequent occupation) in the sixteenth century. The walls on the far side drop down a sheer cliff to the Mtkvari far below, so if you fancy a bout of vertigo, pull yourself up and look straight down.
Vardzia — the cliffside cave monastery, former city, and UNESCO World Heritage site is the biggest reason to visit Akhaltsikhe, and the region more generally. A round trip by taxi from Akhaltsikhe might cost about 60 lari.
Safara — a monastery on a lovely, lush hillside about 12 kilometres along an untarmacked road. Very worth visiting. A round trip by taxi from Akhaltsikhe might cost 15-25 lari.
Akhalkalaki — the center of Armenian culture in Georgia is nearby to the southwest, easily accessible via marshrutka from Akhaltsikhe
Borjomi — every Soviet traveler's favorite Georgian retreat, for its parks, Romanov palace, and internationally renowned natural mineral water
Turkey — You can buy bus tickets to Turkey from the central bus station. Ticket to Posof (city on the Turkish side) is $10 and leaves at 14:30. If you are going to Kars/Ani and have a tight budget, hitching to Kars is a very easy and quick option and can be done in far less than a day. If you leave in the morning, hitching to the Turkish border or to Posof can be quicker than the bus. From the central bus station, ask for the road to the Turkish border (Turetskoĭ granitsy) and start flagging cars about a kilometer from the bus station down the road. The road will pass through the small town of Vale (note: although Vale does not appear to be on the route to Turkey on the Georgian tourist maps found at the tourist information centers, it is indeed on the way). You will be able to walk across the border.