The chance to have an impact on students at such a crucial stage in their education, to try to build a solid foundation and stretch their imaginations, was a challenge he couldn't resist.
Retiring this month as principal of Old Forge Elementary School, Phillips said he never regretted the switch to elementary education.
He had been teaching physical education at the elementary level for just a few years when then-schools Superintendent Claud E. Kitchens encouraged him to try his hand at school administration.
"He had faith in me," said Phillips, who became principal at Paramount Elementary School in 1974.
In 1990, he took over as principal of Old Forge Elementary School east of Hagerstown.
Although he gave up coaching, Phillips said he has stressed its basic principles - preparation, practice, teamwork, character and learning from mistakes - in teaching and running his schools.
In public education, he believes, you need to give attention to students, encouraging them to be their personal best and follow through on commitments.
Growing up in Ridgely, W.Va., he said he had three role models who influenced his vocation and approach to education.
Those role models were his mother, who always valued education; Miss Abbott, a teacher who came to his games and always found something positive to say about his performance; and his high school coach, who taught him the value of sports and not to be a quitter.
"I've been going to a schoolhouse since I was 5 years old," he said. "I just felt it was time to stop and smell the roses."
Retirement will give him more time to devote to his duties as chairman of Washington County Parks and Recreation board and to his seat on Dream Come True executive board, he said.
He's also looking forward to spending more time with his three grandchildren, playing golf and traveling with his wife, Anna Margaret, a kindergarten teacher.