You cannot cross directly into Famagusta from the government-controlled areas of Cyprus. Instead, you must cross the Green Line into the Turkish-occupied territory (known as the de-facto Turkish Republic of Northern Cyprus). The nearest place to do this is from the Strovilia crossing near Agios Nikolaos, which is located inside the British sovereign military base area.
In general, you will need to travel by car. Famagusta has no functional public transportation, and there are no regularly scheduled public transportation links across the Green Line (although private tour operators do operate across the Green Line during peak tourist periods).
Self-drive is the most common way of visiting Famagusta, whether in your own vehicle or a rental car. Bicycle rentals are not available. It's possible to travel on foot, but sidewalks are in very poor condition where they exist. If on foot, be very careful not to accidentally cross into areas quarantined by the Turkish army and/or the UN, as doing so risks arrest.