Anei Kankō and Hateruma Kaiun together have four departures daily from Ishigaki harbor. If you buy a round-trip ticket, be sure to catch the same operator's boat on the way back. Travel time is 60 minutes.
Japan's Southernmost Point Monument (日本最南端の碑) is on the southeast coast of the island. As the local maps say, "Beyond here, the Philippines".
Astronomical Observatory (星空観測タワー), on the southeast corner (300m from the southernmost point). Open every evening except Monday, with a minimal planetarium presentation. The Southern Cross is visible from mid December to June. Many local inns provide shuttle service (free or Y500) to and from the observatory.
Hateruma's local specialty Awanami (泡波) makes its way from the distillery to the local stores only sporadically, and the islanders usually buy it all up within hours. So if you come across anything larger than the tiny 100ml bottles, grab it. Fuka Shōten (冨嘉商店) is a good place to look.