The archaeological site of Delphi is an UNESCO World Heritage site  near the modern village of the same name in Greece. In ancient times it was the site of the most important oracle of the god Apollo. Delphi was revered throughout the Greek world as the site of the ομφαλός (omphalos) stone, the centre of the universe. In the inner ἑστία (hestia), or hearth, of the Temple of Delphic Apollo, an άσβεστος φλόγα (eternal flame) burned. After the battle of Plataea, the Greek cities extinguished their fires and brought new fire from the hearth of Greece, at Delphi; in the foundation stories of several Greek colonies, the founding colonists were first dedicated at Delphi.
There are several companies offering day trips from Athens to Delphi, prices varying from €80 to €100. To save money, you can easily organise your own day trip utilising Athens long distance buses. The journey takes about 3 hours. One-way fare: €16,40 (no discount for return ticket). Seats are numbered.
Buses leave several times a day from Bus Terminal B. Finding this terminal is a bit tricky, as it is not on a main road and local buses do not stop there. As shown on this map, you can take a bus to Liosion Street and get out around number 260. Then walk to parallel street Agiou Dimitriou Oplon. Use Google Maps Public Transport to find buses to 260 Liosion St. See also Athens Guide: Bus Terminal B.
It is recommended to buy bus (return) tickets to Delphi the day in advance, to be sure they are not sold out, as well as to familiarize yourself with the directions to the bus terminal.
Daily times to Delphi from Athens are: 07:30, 10:30, 13:00, 15:30, 17:30, 20:00
From other cities
While the bus to/from Athens (and every city included on the route) is direct; for/from all other destinations, you have to take a bus to the close city of Itea and then wait for the Delphi shuttle. (The wait is not long in season)
Delphi is small enough that walking is really the only means of transportation required. The museum and archaeological site are both in walking distance from the town. Taxis however are readily available.
The region of Delphi is typically Greek countryside: mountainous, with narrow roads. It's nice for hiking and cycling, especially in spring and fall as summer is really hot. Some hostels rent bikes.
There is a lot of touristic stuff to buy.
Many cafés and restaurants are located on the valley side of the main street (Pavlou friderikis str.), and therefore offer a stunning view over this valley. Expect prices of the meals and drinks to be ~15-25% higher than in other Greek cities, very touristic landmarks (such as Plaka-Monastiraki area, in Athens). You'll find traditional Greek cuisine, Italian as well and occasionnally, sea food. Note some restaurants are closed in the evening, even in season.
Visit the nearby Arachova villages and also Parnassos Mountain during the winter season for skiing.