Kayaköy, or Levissi as it was known to its former inhabitants, was a Greek town until 1923, when, after the multinational Ottoman Empire drew to close, governments of Greece and Turkey concluded on a population exchange to become nation states ethnically homogenous as much as possible on the basis of Treaty of Lausanne. According to that treaty, all Greek Orthodox inhabitants of Turkey were to be exiled to Greece, while all Muslim inhabitants of Greece were to be exiled to Turkey in return. When the Greek inhabitants of Kayaköy left for Greece, Muslims from Greek Macedonia were settled in their place. However, Macedonians who were used to large and fertile fields in their former land found this hilly and rocky area with little arable land unfit to live, and abandoned the place in favour of other regions, and thus Kayaköy has become what it is today.
In its heyday, Kayaköy was populated enough to support a local newspaper and several schools and stores, but today there is only a handful of natives living there, mostly in the neighbourhood of Keçiler, 2 km north of the "ghost town" of Kayaköy.
You'll mostly walk around the ghost town.
The ghost town of Kayaköy itself is the main sight. Those not to be missed include the old fountain which dates back to 1888 by the tarmac road, the abandoned church, and the little chapel on the top of the hill (about 20 min uphill walk from the church; follow the red dots from the church), which gives a stunning view of the valley and the sea below, which are located on the other side of the hill that Kayaköy leans against, and therefore is not visible from the town itself.
The church and the streets (or stairs to be more precisely) in its vicinity were declared a "museum" by the Turkish Ministry of Culture with a ticket office in the entrance which require anyone passing through—whether they have the intention to check out the church or not—to buy a ticket which costs 8 TL pp. You may pass on the early and late hours of the day on which the office is unmanned, though.
There are restaurants in and around the town, mainly between Keçiler and Kayaköy.
There is a number of guesthouses (pansiyon) both in and out of the ghost town. There are also some bungalows which start from 20 TL a night.