Go to the Taipei Main Station which serves both regular trains and the MRT system in different areas. (You cannot directly transfer from one system to the other.) Take the Ziqiang Train to Hualien, which is about a 3 hour trip from Taipei.
The bus station heading south to Taitung is to the left as you come out of the Hualien Train Station. Buses travel down the scenic Highway 11, and usually transfer in Jing Pu, near Big Harbor and the famous Big Red Bridge. Try the homemade dumplings at the noodle shop in Jing Pu while you're waiting.
Trains travel down the East Rift Valley National Scenic Area and are perhaps the most convenient way to travel from Taitung to Hualien. Spend a few days slowly making your way down this area. The sightseeing is the best in Taiwan for those that love nature and beauty.