This site was selected in 1866 as a good cemetery location in the general vicinity of various Civil War battlefields and scenes of action related particularly to the Peninsular Campaign of 1862 when General George B. McClellan was moving toward Richmond, the Confederate capital. The cemetery lay adjacent to the spot on the the 1781 Battlefield where the British had surrendered to General Washington.

There are 1,596 marked graves in the cemetery. Of the total of 2,204 burials, 747 are of known persons and 1,436 unknown. Those buried here were for the most part Union Army soldiers, although 10 Confederate soldiers and three wives are also identified. In an 1868 inspection made by the U.S. Army, it was reported that:

The interments number 2,180 of which number 11 officers, 716 white soldiers, four sailors, six colored soldiers, and eight citizens are known and two officers, 1,422 white soldiers, five colored soldiers, and 6 citizens are unknown. Besides the burials at the cemetery, bodies were removed from Williamsburg in James City County, and altogether from twenty‑ seven different places in the surrounding country, within a distance of fifty miles.

Those nearby points included White House Landing, King and Queen Courthouse, Cumberland Landing, West Point and Warwick Courthouse.