Rousseau resigned his seat in the Kentucky state senate to recruit for the Union. In September 1861, he mustered in as colonel of the 3rd Kentucky Infantry. In October 1862, Rousseau was promoted to brigadier general, and less than two weeks later, to major general. He commanded a brigade at the battle of Shiloh, and a division at the Battle of Perryville. Rousseau was transferred from the Army of the Ohio to the Army of the Cumberland, with which he served in the Battle of Stone's River, in the Tullahoma campaign, and in the last of the Chickamauga campaign (he was not present at the battle of Chickamauga). From November 1863 to November 1865, Rousseau commanded the districts of Nashville and Tennessee, before he resigned in 1864 to take a seat in Congress.