Born and raised in Richmond, Virginia, Armstead Walker came from a upper-middle class African-American family who ran a brick contracting business. After graduating high school he helped his brother run the family brick contracting business, worked as a postal worker and became husband to Maggie Lena Walker, President of St. Luke Penny Savings Bank.
Armstead Walker Jr. was born in December, 1860 in Richmond, Virginia. His family operated a brick contracting business in Richmond and was an upper-middle class family. He attended the segregated Richmond Public Schools, where he graduated from in 1875. He began working for his father's construction business in Richmond, learning the ins-and-outs of brick contracting on such buildings as the Leigh St. Armory while becoming well-connected with the upper class of Black Richmond.
During this time, Armstead would meet Maggie Mitchell, a teacher for Richmond Public Schools. After a courtship, they married on September 14th, 1886 at the First African Baptist Church. The Walker family grew when Maggie L. Walker gave birth three sons born between 1890 and 1897, though it was not without its hardship as their second son would pass away before his first birthday. Armstead would also have a distantly related niece, Margaret "Polly" Anderson come live with his family in the mid-1890s.
Armstead and his brother Andrew both continued running the family brick contracting business and Armstead also became a mail carrier in 1901, showing his connections in Richmond because mail carrier positions were given through the patronage system during this era. Armstead would also become a part of several organizations and social clubs throughout Richmond, such as the Richmond Athletic Social Club.
On June 20th, 1915, Armstead Walker would be unexpectedly killed at the hands of his oldest son Russell. Mistaking him for a burglar, Russell used a pistol to fire through a screen on the back porch killing his father. Though he stood trial for murder, Russell was found not guilty but the consequences of his action and Armstead's death would still be felt for the Walker family and the Jackson Ward community.