Milton was born in 1947 in East Texas and raised locally. He has been employed as a college teacher, cattleman, and political consultant. He has four grown children as well as four grandchildren. He is very much a partisan of the South.