Gavin told council members that cutting the grass at the home and doing other chores is too much for him and believes it is time to move.
Gavin wants to sell the home, but did not mention an asking price at Monday's meeting. Gavin said someone who specializes in appraising historic homes probably would have to determine the price.
Because the town does not have the money to buy the home, city council members are considering having an organization known as the Trust for Public Land identify sources of funding for the purchase.
A variety of possible funding sources, including state grants and creative financing proposals, could be used to pay for Happy Retreat, said Debi Osborne, Chesapeake field office director for the Trust for Public Land.
Gavin said he wanted to present the idea to the council to see how they felt about it.
Council members supported the proposal.
Mayor Randy Hilton said it is a rare opportunity because few places in the country can boast of having a connection to the Washington family.
Council member Matt Ward said because Happy Retreat is close to the Huntfield development, it could complement a proposed bike path serving Huntfield and the city.
"It would be really sad if Charles Town did not own a Washington home," said council member Geraldine Willingham.
Charles Washington was the brother of President George Washington. Charles Washington built Happy Retreat in 1780 and lived there until his death in 1799, Gavin said.
The two-story home has nine bedrooms, and the front view of the home is used on the city's stationery. There are six other Washington homes in the area, including Beall-Air, Claymont Court, Harewood, Blakeley, Cedar Lawn and Richwood Hall.
Charles Washington - credited as the founder of Charles Town - took 80 acres he owned and divided them into lots. Happy Retreat was just a short distance away from his real estate office on Lawrence Street, which he probably reached by horseback, Gavin said.
"He was the original developer," Gavin said.
When Charles Washington laid out the town, he set aside the four corners at the intersection of George and Washington streets, the main intersection in town, for public use.
The city is honoring that part of the town's history by building a simulated brick and cobblestone surface at the intersection as part of the town's $7.1 million revitalization.