W C Handy was born in Florence, Alabama, the son of Elizabeth Brewer and Charles Barnard Handy. His father was the pastor of a small church in Guntersville, a small town in northeast central Alabama. Handy wrote in his 1941 autobiography, Father of the Blues, that he was born in the log cabin built by his grandfather William Wise Handy, who became an African Methodist Episcopal minister after emancipation. The log cabin of Handy's birth has been preserved near downtown Florence.

​

Growing up he apprenticed in carpentry, shoe making and plastering. W C Handy was deeply religious, and his musical style was influenced by the church music he sang and played as a youth. It was also influenced by the sounds of the natural world. He cited as inspiration the sounds of "whippoorwills, bats and hoot owls and their outlandish noises", the sound of Cypress Creek washing on the fringes of the woodland, and "the music of every songbird and all the symphonies of their unpremeditated art".

​

W C Handy's father believed that musical instruments were tools of the devil. Without his parents' permission, He bought his first guitar, which he had seen in a local shop window and secretly saved for by picking berries and nuts and making lye soap. Upon seeing the guitar, his father asked him, "What possessed you to bring a sinful thing like that into our Christian home?" and ordered him to "take it back where it came from", but he also arranged for his son to take organ lessons. The organ lessons did not last long, but Handy moved on to learn to play the cornet. He joined a local band as a teenager, but he kept this fact a secret from his parents. He purchased a cornet from a fellow band member and spent every free minute practicing it.