Located across the bay from Aomori, Mutsu merged with two neighboring towns (Ohata Town and Kawauchi Town), and one village (Wakinosawa village) in 2005 and its city limits now cover most of the peninsula. The former Mutsu city is famous as the site of Osorezan (Mt. Fear) one of the "3 most spiritual places" in Japan. Mutsu also serves as the central transport hub of the peninsula, so it's a convenient place to base yourself while exploring its attractions.
Mutsu is well connected by Shimokita standards.
The JR Ominato line from Noheji (on the Tohoku main line) passes through Shimokita station (下北駅) in the southern suburbs of Mutsu.
From Tokyo station it takes roughly five hours to reach Shimokita. You will need to take three trains: The Tohoku Shinkansen Hayate to Hachinohe, the Aoimori Railway to Noheji, then the JR Ominato local to Shimokita station. This trip will make your wallet almost ¥17,000 lighter each way.
The Japan Rail Pass and JR East Rail Pass fully cover the trip, including the segment on the Aoimori Railway between Hachinohe and Noheji. Note that seats on the Hayate bullet train require a seat reservation (free with a Pass).
Buses from Mutsu's central bus terminal connect to various points in the peninsula, including Ohata (35 min, many daily), Oma (1.5h, 8 daily) and Mount Osore (30 min, 4 daily). No direct services to the Yagen Valley though, you'll have to connect through Ohata in time to catch the single daily bus.
Local bus services provide transportation around Mutsu, but the center is small enough to walk.
Central Mutsu has a number of quaint bars and restaurants.
Mutsu has a wide selection of reasonably cheap lodgings (¥6,000 with two meals), and the tourist information office is glad to help you book.
Mutsu Tourist Office (Masakari Plaza, tel. +81 0175-22-0909), near the bus terminal, is helpful and can provide basic information in English, not just for Mutsu but the entire peninsula. They will also book accommodation for you.