Before the Federal government had fully addressed the question of contraband policy in the summer of 1862, the Union Navy operating in the St. Johns had already adopted a practice of not returning fugitive slaves and establishing protected contraband communities. The Navy faced a challenge feeding the significant numbers of contraband and loyal Unionist whites they picked up, sometimes in groups numbering forty to sixty people, who could quickly deplete the ship's limited food and water supply as well as crowd the limited space onboard. In response contraband had to be sent to larger established Union bases in the South, or if that was not possible, placed ashore at improvised colonies such as Pilot Town. As early as June 1862, the Navy began delivering Unionist refugees and contraband to Pilot Town. As the colony quickly grew in size, its population included entire families and came to provide a relatively stable environment from which men could achieve employment with the military and women could earn a wage as servants or cooks. Additionally, small amounts of trade were recorded with the Union ships. An estimated two hundred twenty contraband passed through Pilot Town in the summer and fall of 1862 with a consistent population of one hundred persons by the close of the year. Although the colony could provide some of its own food in the form of corn and fish, a Union commander noted that "these colonists are all in much want of food and clothing, the latter of which it is entirely out of my power to supply."
Providing for the welfare of the colony at Pilot Town also meant ensuring its protection. Both conventional and guerilla Confederate units tried to prevent contraband from reaching the Union blockade fleet by confiscating or destroying boats belonging to Unionists along the river as well as attacking the Navy base at Mayport. To safe guard Pilot Town in areas out of range for the Union gunboats, both sailors and community members established and manned lookout posts with obstacles at key points of access to Batton Island. As further insurance against attacks, the Navy used axes to cut down the sixty-foot bridge across that connected Batton and Fort George Island and later launched a reconnaissance expedition on Fort George Island in December 1862. However, the Union protection of Pilot Town was more than just humanitarian; it had a political purpose as well. Naval commanders believed that if they could promise protection for Pilot Town, it would serve as a "nucleus for [refugees] to rally around" encouraging an "exodus" including those "who might otherwise be compelled to give their services to the Rebels." For this reason the Navy desired a permanent Union Army force be stationed at Pilot Town. However by the close of 1862, an increased Confederate presence in the area and the drain caused by Pilot Town's large population on the Navy's limited resources resulted in the colony's occupants being removed to larger military bases. Pilot Town was periodically revived as a contraband colony until the end of the war as well as serving as a small Union Army signal station between gunboats at the river mouth and Jacksonville.