Chiayi is best known as the access point of the scenic mountain area of Alishan.
Chiayi is also served by mainline TRA trains and the scenic narrow-gauge Alishan Forest Railway to Alishan, which both leave from the same station, but have separate ticketing counters. Long-distance buses leave from the terminal immediately outside the station, but local buses (notably those to nearby hot spring town Guanzihling) use the Zhongshan Rd bus terminal a few hundred meters down the street.
The Chiayi Bus Rapid Transit (BRT)  system has three lines, with the main line linking the HSR and TRA railway stations. The trip between the two takes around 30 minutes ($40, exact change - no charge if going directly between the HSR & TRA stations), but note that the BRT stop is on the rear side of the TRA station, so you need to take the pedestrian bridge to cross to the other side. The location of the TRA station is not obvious from the BRT stop. If coming from the direction of the HSR railway, after you get off at the stop, cross to the other side of the street at the intersection you have just passed. The "back station" area is under construction - bear to the right and you should find the pedestrian bridge.
Buses have announcements and signage in English as well as Chinese.
Taxis are plentiful, especially around the train stations, but no English is spoken.
Buses to Alishan costs $250NT one way, and can be bought at the bus station next to the train station. The ride takes 2.5 hours and can be quite windy.
Chiayi Park and Botanical Gardens are a lovely way to spend the afternoon. The Botanical Gardens provide pleasant trails that weave you through a large variety of plants and trees native to the area. Ponds hold koi, turtles, and frogs. You can even see hints of a traditional cemetery through the trees in some places. Chiayi park is well-maintained and serves as a center of activity for local adults and children.
Zhongshan Road (中山路) is Chiayi's main shopping street. It stretches from the train station to a fountain (噴水) at Wenhua Road -- although the water is usually turned off.
One specialty chain store that's located on Zhongshang Road is the "Guang Nan" (光南), which caters in small electronics and other media-related goods. Catch a good deal on cables and computer accessories here.
There are two night markets in Chiayi. One is the Wenhua night market on Wenhua road (just hawker stalls by the side of the road, weekends have bigger selection). The other is the Carrefour Night Market next to the Carrefour on Bo Ai Road (closed on Mondays).
Chiayi is famous in tourist guides for turkey rice (鶏肉飯), a popular dish of turkey strips and a savory sauce ladled on the rice. Most people call it "chicken rice" but it's really turkey meat. Go figure. At any rate, it's available in practically every eatery town, at prices starting from $20.
Chiayi has countless tea stalls and quite a few KTV lounges, but virtually no bars in the Western sense.
There are a large number of mostly identical motels just outside the TRA Chiayi station.