"I'm asking that the city conform to the building code that they wrote and that everybody else has to conform to," he said. "I want that building fixed up, and the only way they can do that is to remove it."
Walker's suit lists 20 complaints of building code violations against the Tristate building, which the city bought in late 1996 for $120,000.
Among the alleged violations are missing windows and window screens, visible cracks, weeds and vegetation above allowable height, and missing gutters and downspouts.
The suit also contends that rains and wind deposit chipped paint and eroded cement on Walker's property and that bricks fall intermittently from the building into Walker's yard.
City Administrator Bruce Zimmerman would not comment directly on the lawsuit, but he said the Hagerstown City Council hopes to put the Tristate building to productive use.
"We're hopeful the building can be revitalized as part of an arts and entertainment district," he said.
Zimmerman said the city does basic maintenance work on the building and added that he does not believe anyone has complained about building code violations.
"I'm not aware of any violation," he said.
Walker has been engaged in a nearly year-long battle with the city over the property.
The Hagerstown Preservation Design District Commission denied his request in April for a permit to demolish the home that has been in his family since 1870. The commission determined that the house was in excellent physical condition and contributes to the historical significance of the downtown historic district.
"They think of everything as historic. It's not," Walker said.
The suit requests action within 30 days and $30,000, the cost of maintaining the house for three years.