Caldwell mustered in as colonel of the 11th Maine Infantry in November 1861. He was quickly promoted to brigadier general in April 1862. He served in the Peninsula campaign in command of his brigade. Although absent at Antietam, he was wounded twice in the Battle of Fredericksburg, and again at the Battle of Chancellorsville. He commanded the II Corps temporarily at the battle of Gettysburg. However, in the following year, in March 1864, he was relieved of his duty with the Army of the Potomac. He mustered out in January 1866.
