Professor Elder was born in Oconee County, Georgia. He hardly knew his mother because she lived away in the home for whom she worked. She left him and his only brother with other people. They attended Knox Institute in Athens when they were youngsters. He states in his biography that Mr. John Meeks, a white man provided for them after his mother died. He cared for them for several years in Athens and later in Atlanta, they worked and saved their money and were able to attend Fisk University. He later attended Morgan Park Military School and Cook County Normal in Chicago.
Professor Elder came to Washington County in 1888 at the invitation of a Preacher to take charge of a Washington County School. After two years he built a school which was named the Sandersville High & Industrial School and was later renamed the Thomas Jefferson Elder High and Industrial School.