One of Sanford's visits to his mistress was in 2008 during a taxpayer-funded trip to South America that was supposed to be an economic development mission. The other was in June, when Sanford disappeared from the state and left no one in charge.
The rebuke still requires majority votes in the House and Senate. Those likely won't happen till next year.
In a statement minutes after the vote, Sanford thanked committee members for voting no on impeachment and said he is ready to focus on the next legislative session, which starts in January.
"As we've consistently said, this administration has tried to be a stalwart ally of the taxpayer, and will remain so for the next 13 months," said Sanford, whose wife filed for divorce last week, citing adultery. His second and final term ends in January 2011.
The governor has been under scrutiny since June, when he returned from a secret trip to Argentina and tearfully revealed his affair with a woman named Maria Belen Chapur. His staff had told reporters he was hiking the Appalachian Trail.
Probes of his travel and campaign spending led to more than three dozen ethics charges and the potential for $74,000 in fines. Lawmakers considered a handful of those charges in reaching their censure decision, saying the governor violated the public's trust in part by using state planes for personal and political trips.
Sanford is the first South Carolina governor to face censure. Only eight U.S. governors have been removed by impeachment, and the only two removed in the last 80 years each faced criminal charges.
Some lawmakers still said Wednesday Sanford should lose his job.
State Rep. Greg Delleney, R-Chester, the most vocal proponent of impeachment, said Sanford "through his own deceitful behavior has lost all moral authority."
Among those trying to keep the impeachment effort alive so all 124 House members could vote was Rep. Todd Rutherford, a Columbia Democrat who said a regular state employee would be fired for disappearing from the state for five days as Sanford did.
"This is the way that impeachment should be handled," Rutherford said. "This is important enough ... that every member of the House should get a vote on it."
That's not likely. Rep. Jim Harrison, R-Columbia, chairman of the Judiciary Committee, said the impeachment resolution could be revived with a majority vote in the House, which rarely happens.
"It's as dead as dead can be," said Rep. Bakari Sellers, D-Denmark, who voted for impeachment.
In addition to Sanford's June disappearance, which the rebuke calls a dereliction of duty, lawmakers also focused on a state-funded trip in 2008 during which the governor's longtime friendship with Chapur turned physical.
After he revealed the relationship this summer, Sanford reimbursed the state for part of the prior year's trip. He has maintained that he traveled to Buenos Aires for economic development meetings following a dove hunting excursion elsewhere and says the meeting with Chapur was an afterthought.
Critics contend Sanford steered the Commerce Department to get him to Buenos Aires to see Chapur and questioned how seriously any state business was taken while he was there.