As well as a library, the Atheneum hosted speakers. In August of 1841 the Atheneum was the site of Nantucket's first anti-slavery convention. Over the three day period, several speakers stood at the podium. Among them were Frederick Douglass and William Lloyd Garrison.
During the second Nantucket anti-slavery convention in 1842, abolitionist Rev. Stephen Foster lambasted the clergy for their inaction against slavery, calling them "a generation of vipers" leading a "brotherhood of thieves." The reaction was swift. The next evening, asn William Lloyd Garrison and Frederick Douglass were getting ready to speak, an angry mob approached the Atheneum. Yelling and throwing eggs, stones, and bricks, the convention was forced to change venues. Douglass gave several subsequent speeches at the Atheneum.
The current Nantucket Atheneum building opened in 1847 after an 1846 fire destroyed all of downtown Nantucket, including the old Atheneum and its over 1,200 titles. The current building was constructed at the same location. It is described as "The handsomest building rebuilt after the fire....Perhaps the most monumental building on the island." In 1900 it became a free public library.
The Nantucket Atheneum is a contributing property to the Nantucket Historic District, added to the National Register of Historic Places on December 13, 1966 and designated a National Historic Landmark on November 13, 1966. Link to the nomination.