Kutaisi is the traditional rival of Tbilisi for capital status, and has since the days of the Golden Fleece been considered the capital of Western Georgia (then Ancient Colchis). It remains Georgia's second largest city, but to the irritation of the proud locals, does not come even close to Tbilisi's present day size and wealth. Since the Georgian government decided to move their parliament to Kutaisi, there has been a lot of work on restoring streets, buildings, parks and monuments and the city has become much safer.
But a visit here is nonetheless near mandatory to see the magnificent Bagrati Cathedral, and Gelati Monastery, both of which are UNESCO World Heritage sites, and have commanding views from the mountain slopes over the city and the Rioni river.
New, cheap flights will be instituted from 9/27/2012 to and from Kiev, Warsaw and Katowice by WizzAir.
All trains from and to Tbilisi Central leave from and arrive at the new Kutaisi I station. There is a daily express train (#18) from Tbilisi departing at 8:50AM and taking about six hours to reach Kutaisi I at 3:00PM. A ticket with reserved seat costs minimum 5 GEL (ca. 2.50 USD). Also stops in Mtskheta (old capital of Georgia), Gori (birth place of Stalin) and Zestafoni. Nice journey.
A local but fast train runs from Kutaisi I to Batumi (listed, confusingly as 'Makhinjauri' in the timetable), leaving at 9 AM, arriving 3 hours later. Cost GEL 4, payable on the train. Preferable to the bus.
Train from Skaltubo are running from Kutaisi II station, which is located in the southwest of Kutaisi.
There are more night and day trains running from Tbilisi Central to Rioni station, a few kilometers in the south of Kutaisi.
More info 
(City bus 101 runs from this station to central Batumi and on to the border station with Turkey; no need for any other transport.)
Mini-buses (marshrutkas) and full 54-passenger buses depart regularly for Kutaisi from Didube Bus Station located at the Didube metro stop in Tbilisi. Marshrutkas typically depart once an hour; large buses every three hours. Typical fare is 10 lari. Travel by marshrutkas takes roughly 3.5 hours; larger buses take slightly longer due to more frequent stops.
Already mentioned, but one simply cannot have too much of Georgian wine.