Because of the home's strong connection to city history, Gavin said he wants the city to have the house.
During the Charles Town City Council meeting on Sept. 15, Gavin proposed that the city purchase the home from him.
Mayor Randy Hilton and council members praised the idea, saying it presents a rare opportunity for the town.
Although Hilton said that no specific uses for the home have been discussed, at the least it could be opened to the public for viewing. Hilton said he was sure that opening a Washington home to the public would be a tourist draw.
According to local history accounts, George Washington visited his brother periodically at the home.
"I don't think we're going to have any problems finding a use for it," Hilton said.
With its 12 acres, Happy Retreat could be a valuable source of green space for the city, not to mention other possible uses such as a museum, Gavin said.
Because the city does not have the money to purchase the home, council members are considering having an organization known as the Trust for Public Land identify sources of funding for the purchase.
There are a variety of funding sources, including state grants and creative financing proposals, that could be used to pay for Happy Retreat, said Debi Osborne, Chesapeake field office director for the Trust for Public Land.
Gavin has not asked for a specific price for the property. He said someone who specializes in appraising historic homes probably would have to determine the price.
Charles Washington built two wings for the home. Other than a breezeway between the two wings, that was the extent of the house when he owned it.
"He was not primarily concerned with fine living quarters, he was obsessed with founding a town," according to the book "Between the Shenandoah and the Potomac: Historic Homes of Jefferson County, W.Va."
"Over the next 19 years, the construction of the planned center portion of Happy Retreat never took place. Evidently his wife Mildred was a woman who knew how to make do with what she had," according to the book, which was produced by the Jefferson County Historical Society.
The home, stately inside and out, has marble fireplaces and decorative medallions in the ceilings.
The center portion of the house was built in 1837 by Isaac Douglass, a Circuit judge in Jefferson County at the time, Gavin said.
Douglass would have presided over abolitionist John Brown's treason trial in Charles Town in 1859, but fate changed that.
Douglass was killed when he was thrown from a horse, Gavin said.
Francis Drew owned Happy Retreat during the Civil War, Gavin said.
At one point, Drew flew a Confederate flag from the front of the house, drawing the attention of Northern troops.
"The Yanks came up, surrounded the house and made him take it down," Gavin said.
Past owners of the house included R.J. Funkhouser, a wealthy businessman who was considered to be one of the most influential people in the county in the mid-1900s.
Funkhouser owned Happy Retreat from 1942 to 1950 and conducted a major renovation of the white brick house.
Other features of the property include a pre-Revolutionary War smokehouse and a stone kitchen dating to 1768.