In 1796, the government purchased a 125-acre parcel of land bounded by the Potomac and Shenandoah rivers from the heirs or Robert Harper. Construction of the "United States Armory and Arsenal at Harpers Ferry" began in 1799. Three years later full-scale production of arms commenced. Muskets, rifles and, after 1805, pistols were all manufactured here. By 1810, annual production of arms averaged about 10,000.
But from the very beginning, the Harpers Ferry Armory labored under a deficiency of waterpower and charges of mismanagement. After more than four decades of piecemeal funding and outright neglect, plans for the site’s complete renovation were drawn up in 1844. Between 1845-1854, seven new workshops were erected, 121 new machines were installed, the Armory Canal was enlarged, and heavier millwork including the installation of new Boyden turbines was completed. The new buildings consisted of brick superstructures with cast-iron framing, sheet-iron or slated roofs, and large arched portals and windows, all erected upon heavy stone foundations.
This extensive reconstruction created a more integrated physical plant, whereby the flow of work from one stage of arms production to another was greatly facilitated. These buildings became known collectively as the "U.S. Musket Factory." Compared to 1802, when the Armory employed just 25 men, by 1859 the workforce had grown to about 400.
With the Civil War there came destruction. When Virginia passed an ordnance of secession on April 17, 1861, the Harpers Ferry Armory became an immediate target. Lt. Roger Jones of the U.S. Army defended the Armory on the evening of April 18, 1861, with 50 regulars and 15 volunteers. In nearby Charles Town several companies of Virginia militia – 360 men in all – assembled and advanced toward Harpers Ferry. Jones, outnumbered and unable to obtain reinforcements, set torches to the Armory and Arsenal buildings before retreating across the Potomac River.
Further destruction came with Confederate occupation of Harpers Ferry in the spring of 1861. Southern forces confiscated the Armory’s ordnance stock, machinery, and tools before burning many of the remaining Armory buildings. By war's end, only the Armory's fire engine and guard house – John Brown's Fort – remained intact. Today, the Armory Grounds are almost completely covered by railroad track embankments.