Hilton on Tuesday cited family illness as his reason for not running. He said leaving elected office after 19 years will be difficult.
Hilton has been mayor for nine years and was a city council member for 10 years.
"It's hard to say goodbye," Hilton said.
When Hilton first ran for mayor, he was not real excited about the job. Hilton said he decided to run because some people had encouraged him to do so in hopes he could do something about drug trafficking in town.
Around 1997, drug dealing was so bad in the area around Evitt's Run Park that police had to clear out drug traffickers so children could attend special events in the park, police said.
There was gunplay and shootings. In a fatal shooting in 1993, the victim was found lying along South West Street clutching a chunk of crack.
After Hilton was elected, he pushed for a study of the police department by the International Chiefs of Police. In the study, the organization came up with about 32 recommendations, including hiring a chief of police.
At the time, the highest position in the department was captain and the department was overseen by the mayor.
Hilton selected Mike Aldridge to become police chief and take over the department.
Aldridge, a former federal Drug Enforcement Agency official, was able to bring a federal DEA team to the city to investigate the trafficking.
The DEA team conducted a five-month investigation that resulted in 22 arrests.
Following the action, it was a different town, Aldridge and residents said.
Aldridge, who was police chief for eight years, retired from the department Sunday.
"My goal was to leave the city in a better place. I hope I've done that," said Hilton, a retired human resources specialist.
Hilton said he is satisfied with other efforts that have been started in the city during his tenure, such as raising funds to renovate Charles Washington Hall, a historic building at the intersection of George and Washington streets which was named for the town's founder, Charles Washington.
The city also wants to extend the downtown revitalization project east along Washington Street.
The revitalization initially extended from West Street to Samuel Street along Washington Street and from the Old Opera House to Congress Street on George Street. The work included new sidewalks, new streetlights, tree plantings and a historical interpretive area next to the Jefferson County Courthouse.
"I fully intended to continue because we have a lot going on," Hilton said.
Hilton said he is proud of the new police station at the corner of Liberty and Charles streets.
During his tenure, Hilton said he believed a visitor's center was needed to capitalize on Charles Town's vast history.
Apart from a brochure that shows visitors how to take a walking tour of the town's attractions, tourists are largely on their own when discovering the town, Hilton said.
Following an agreement between the Charles Town Building Commission and a private group, an old fire hall along George Street next to the Jefferson County Courthouse was renovated and part of the building was made into the Charles Town Visitors Center.
Besides offering tourist information at the center, officials said they planned to continuously showcase various works of art in the visitors center.
"We should all be proud of the great service that Mayor Randy Hilton has given our community. The results of his leadership are real - safe streets, a revitalized downtown, smart growth and a strong future," Ward said.