Getting around the city should not be a problem as it's small enough to be covered on foot.
The tricky part is getting around the desert as there is no public transportation and it can get expensive. You can either :
Get a local on the main square to drive you by car or moto for a prohibitive price of at least 20,000 COP one way, which is almost as much as what you'd pay for a tour.
Go on a tour : prices start at 50,000 COP. The guide will drive you to all the main attractions of the desert for 3-4 hours. There will most likely be someone waiting for you at the arrival of your bus. Always negociate.
Go walking : Distances are small enough if you're not carrying a big backpack but the heat can make this tiring. Avoid midday heat and carry a lot of water. Town to the observatory is 6 km on paved road and from there to Los Hoyos is another 6 km.
Museo Paleontológico, (On the Main Square). M-F : 7:30am to 1pm and 2pm to 5:30pm /Sa-Su : 7am to 6pm. The local findings are divided in 3 rooms : reptiles, mammals and environment. It houses remains of a megaterio, a giant sloth that reached 5 meters. Check out the real sized model in the middle of the plaza.2,000 COP. edit
Astronomy, ☎ 310-465-6765 / (8)-879-7584 (email@example.com). Passionate and knowledgeable astronomer Javier Fernando Rua Restrepo operates the desert observatory. Sessions start as soon as the sun sets and include observation with a modern telescope. You're almost guaranteed to find him there on week ends but give him a call if you plan to visit during the week. 10,000 COP per person. edit
Prices in the desert are a bit higher than in Villavieja but still very reasonable : bolsa de agua : 500 COP / gaseosa : 1,500 COP / beer : 2,000 COP / meal : 5,000 COP.
Conservas del Desierto, Carrera 5 No 5-78, ☎ 311 883-1570 / 311 871 1256 (firstname.lastname@example.org). Sells more than 21 products from the Nopal cactus : sweets, skin lotion, shampoo, arequipe and even cactus "wine".edit