Rosser departed from the U.S. Military Academy two weeks before his expected graduation to join the Confederate army. He participated in the Battle of Blackburn's Ford, and the subsequent action at the Battle of First Manassas. In May 1862, Rosser served with distinction at the Battle of Yorktown, and soon after, he was wounded at the Battle of Mechanicsville. Thus, Rosser earned a promotion to lieutenant colonel in command of the 5th Virginia Cavalry under J.E.B. Stuart. He participated in the battles of Catlett's Station, South Mountain, and Antietam. In December 1862, Rosser was wounded at the Battle of Kelly's Ford. After his recovery, he returned to service in the battles of Chancellorsville and the subsequent Gettysburg campaign. Rosser was promoted to brigadier general, in command of the Laurel Bridge, which he led into the battles of Wilderness, Trevilian Station, and finally, in the Petersburg campaign. After the death of J.E.B. Stuart, Rosser took over command of the Confederate cavalry, after which time he led his command into action at the Battle of Cedar Creek. In October 1864, Rosser's forces participated in the Valley campaign, fighting back the reorganized Union cavalry under Philip Sheridan. He commanded the Confederate forces at the Battle of Tom's Brook on October 9,1864. In April 1865, Rosser attempted to outmaneuver the persistent Union cavalry during the course of the Appomattox campaign. He commanded the Confederate side at the battles of Amelia Springs, and High Bridge and was captured at the Battle of Hanover Court House, and paroled after the end of the war.