There are a few (3?) buses a day from Agadir to Taroudant (Note: buses from Marrakech advertised to be going to El Jadida goto Agadir really, you will need to change there). The bus may drop you off quite a way from the town centre, so be prepared to walk or use a petite taxi.
Taroudant is a medium sized town. Petite taxis are available, as usual, to get you around. It takes about an hour to walk across the whole of the town centre.
There is a cheap hostel on the South(?) side of the main square in the town centre (I'll add the name later!). With a restaurant in front of it.