Southern Armenia. Although the village of nearby Agarak is technically the crossing point into Iran, this is the last real town and the gateway for those entering. The sheer jagged brown mountains here jut up from the thick waters of the Arax River. The old mud adobe architecture of the town is crumbling, much of it replaced with Soviet, but parts of it remain for those who want a taste of a town with a very middle eastern flair. There are a few churches, 2 of them are in very good condition and absolutely covered in frescoes inside. A series of fortress/fortifications above the city are some of the most recent in Armenia, only a couple of centuries old.
Minibus/Van (marshutni), bus, car, taxi from all points north in Armenia, or the border crossing in Iran.
There is at least one marshrutka from Yerevan to Meghri per day, which takes about 8 hours and then continues all the way to the Iranian border in Agarak. It leaves early in the morning whenever it is full. As of a Sunday in early September 2014 it was full at 8.20 and left that time. Experiences may differ, but it's better to be there quite early. The ticket prices as of 2014 is 6000 AMD per person.
Unlike in the past, marshrutkas from Yerevan don't leave at Kilikia bus terminal anymore. They now leave at the Kayakan terminal, which is a small station south of Yerevan main train station or Sasuntsidavit metro stop. Taxi drivers will know the way, by foot it might be pretty hard to find.
By foot or car/cab.
The old churches, the older sections of town which surround them, the fortresses, the Arax River (but keep your distance!). There are seven Prime Butterfly Areas designated for the Meghri region, which provide opportunities for observation of over 140 species, including Dingy Swift, Southern Swallowtail, Dawn Clouded Yellow, White-edged Rock Brown, Knapweed Fritillary, Hyrcanian Hairstreak, and number of others (for details see Butterfly Conservation Armenia).
Same options as getting in - plus hitching with an Iranian trucker!