As early as the Spanish occupation of the San Francisco Bay, the southern shore of the Golden Gate was considered strategically crucial to defending the harbor. In 1851, shortly after the American occupation, the War Department ordered the construction of casemate fortifications on either side of the Golden Gate. The fort on the southern shore was assigned the highest priority; from a military perspective, the new fort would constitute the cornerstone of San Francisco Bay defenses and—by extension—the entire Pacific coast.
The U.S. Army Corps of Engineers began work on Fort Point in 1853. Due to the relatively short range of artillery at the time, plans specified that the lowest tier of artillery be as close as possible to water level so cannonballs could ricochet across the water's surface to hit enemy ships. To accomplish this, the ninety-foot cliff at the construction site was blasted down to a mere fifteen feet above sea level. The new structure had three tiers of casemates (vaulted rooms housing cannon), seven-foot thick walls, and a barbette tier on the roof with additional guns and a sod covering to absorb the impact of enemy cannon fire. While more than thirty such forts existed on the east coast, Fort Point was the only one of its kind on the Pacific.
With the 1861 outbreak of the Civil War, the Army mounted the first fifty-five guns at Fort Point. By October 1861, there were sixty-nine guns in and around the fort, consisting of 24, 32, and 42-pounders, as well as 8 and 10-inch Columbiads. During the Civil War, as many as five hundred men from the 3rd U.S. Artillery, the 9th U.S. Infantry, and the 8th California Volunteer Infantry were garrisoned at Fort Point. Stationed several thousand miles from the major theaters of combat, the men spent their days in a routine of drills, artillery practice, inspections, sentry duty, and maintenance chores. Enlisted men bunked twenty-four to a casemate on the third tier; officers had single or double quarters on the floor below. To supplement coal fuel, soldiers gathered driftwood from the shore for fuel. Though prepared for attack, Fort Point never fired its guns in defense during the Civil War.
After the war, the Army installed powerful 10-inch Rodman guns in the lower casemates; these could fire a 128-pound shot more than two miles. At its greatest strength, the fort mounted 102 cannon. In addition, the fort had "hotshot" furnaces, which allowed iron cannon balls to heated red hot, loaded, and fired at wooden ships to set them ablaze. Despite such technology, Fort Point never mounted the 141 cannon that its planners envisioned.
Advances in artillery during the Civil War demonstrated that brick forts similar to Fort Point—including Fort Sumter in South Carolina and Fort Pulaski in Georgia—were easily breached by rifled artillery. Soon after the war, the army reworked its coastal defense strategy and, in 1870, some of the fort's cannon were moved to East Battery. Though no longer considered the guardian of the bay, Fort Point nevertheless remained important to the army. Among other uses, Fort Point held a machine and welding shop after its closure.