Plasterer and father of Paul Laurence Dunbar, Joshua was born enslaved in Kentucky, perhaps in Garrard County. Historians know little about his early life. Joshua trained as a plasterer while enslaved. His master allowed him to work away from his home plantation and keep some of his earnings. With the assistance of Quakers, he eventually fled Kentucky, venturing to Upper Canada through the Underground Railroad network. He returned to the United States after the creation of the United States Colored Troops in 1863. Joshua enlisted in the 55th Massachusetts Regiment (Colored) under Colonel Norwood P. Hallowell on June 5, 1863. Medical officials discharged him from the 55th Massachusetts on October 28, 1863 for varicose veins, but Joshua soon reenlisted in the 5thMassachusetts Cavalry Regiment (Colored). He served in this unit until the end of the war, rising to the rank of sergeant. He was discharged in Clarksville, Texas, on October 3, 1865. For the remainder of his life, Joshua drew a monthly pension of $25.00 for his service.
With the end of the war, Joshua moved to Dayton, Ohio, where he met Matilda Burton Murphy sometime after 1866. They married on December 24, 1871. Matilda gave birth to Paul, their first son, that next June. Joshua and Matilda had one other child, Elizabeth (1874-1876). Joshua attempted to work as a plasterer but racism left him unemployed for extended periods of time. He and Matilda endured a tense, abusive relationship before she left him around 1873; this relationship set a poor example for Paul’s relationship with Alice Moore. Matilda petitioned for a divorce from Joshua and received one on January 9, 1876. Paul memorialized him and other African American soldiers in “The Colored Soldiers” (1895). Joshua died of pneumonia at the National Home for Disabled Volunteer Soldiers in Dayton on August 16, 1885 and is buried in Dayton National Cemetery.