The oldest building still on Fourth Street, The Marin Hotel was built in 1859 using convict labor from the nearby San Quentin Prison. It was the first brick building in San Rafael, using bricks from the prison quarry. It became the most prestigious hotel in the city. 

On May 26, 1861, the hotel was the residence of State Assemblymen Charles Percy and his political rival Daniel Showalter. In Fairfax, they participated in a duel, to settle their feud over the secession of the South in the Civil War. Piercy was shot in the head and killed, and the Marin Journal reported the "large number of ex-members of the Legislature who came over to disgrace Marin County by conniving at a duel.” 

In 1871, the hotel balcony buckled during a San Rafael Day parade, plunging a brass band onto the hard walkway below, injuring several band members and spectators.

The hotel was sold in 1876, and renamed The Central Hotel. It was completely renovated and boasted of "every comfort and accommodation that can be desired...it is supplied with gas, water and baths."