John Pelham was set to graduate from West Point in 1861, but when he learned that his native state of Alabama had seceded from the Union in January 1861, he was torn about whether or not to stay at the academy. He wrote many letters to family and friends asking for advice and struggling over the decision. "Alabama seems determined to leave the Union…It seems pretty hard that I should toil for four and half years for a diploma and then have to leave without it." Pelham even wrote a letter to Jefferson Davis asking what he should do, but unfortunately his question went unanswered.
Pelham ultimately decided to stay at West Point, although he continued to agonize over his situation. In March of 1861 he was appointed to be a First Lieutenant for the Confederacy, and despite the urgings and anger of some of his family members, he did not initially accept his appointment. Ultimately, the decision was made for Pelham with the firing on Fort Sumter the following month. With the outbreak of war, cadets were called upon to uphold their oaths to the army and the United States, but Pelham refused. He resigned his commission and returned to the South.
Pelham's service in the war is marked by his daring actions at the battle of Fredericksburg on December 13th, 1862. As chief of J.E.B. Stuart's artillery, Pelham took one cannon out beyond Confederate lines at Prospect Hill. Using the cover of fog and low country, Pelham positioned his gun on the left flank of the Union Army. At 10 a.m. he opened fire on the line, and continued firing into Union ranks for an hour. He skillfully maneuvered his gun to avoid returning fire from Union artillery, and successfully delayed the Union infantry attack on Prospect Hill. Pelham was praised by his superiors for his efforts, and was called "the gallant Pelham" by General Robert E. Lee.
The following spring, Pelham was mortally wounded at the battle of Kelly's Ford. He was mourned throughout the Confederacy, and received a posthumous promotion to lieutenant colonel.