Foster entered the Union army in 1861 as Major of the 25th Indiana Volunteers. He was quickly promoted to Lieutenant Colonel after the Battle of Fort Donelson. Then, after the battle of Shiloh, he was elected colonel of the 65th and 136th Indiana Infantry. He commanded a division of cavalry in the army of Ohio during the Kentucky and Tennessee campaigns. In September 1863, he commanded the Union forces at the Battle of Blountsville. He mustered out at the expiration of service in 1864.
