Ernest George Smith was born in 1875 in Baldock, Hertfordshire. His father was John Horace Smith (1850-1895), a labourer. John Horace's parents were John Smith (1820-1898), a gardener and Eliza Bray (1821-1874). Both the Smith and Bray families had deep roots in Hertfordshire. John Horace Smith had two children with Martha Haggar (1845-1929) before marrying her in 1869 in Baldock. Martha, who was five years her husband's senior, was the daughter of William Haggar (1824-1902), the long-serving parish clerk of Norton, Hertfordshire and Sarah Wilsher (1824-1907). In about 1877 John and Martha moved their young family to Walton-on-Thames, Surrey. The 1881 census shows the family living at High Street, Walton-on-Thames. John H. Smith, a thirty year old labourer was the head of the family, living with his wife and six children. While in Surrey, John also worked as a painter and decorator and lamplighter. After the 1881 census, two more children were born.