At last there would be an organization to hold pundits' feet to the fire and bring some accountability to a business known more for hot air than constructive debate.
Well, it didn't take long for a pundit who received a negative rating to use his media platform to return fire.
Today radio show host Larry Elder dedicated his entire weekly syndicated column to criticising the "mostly false" judgement he received for saying as a guest on CNN's political talk show Crossfire that "all three levels of government, federal, state and local" is taking "almost 50%" of American's money, compared to only 10% in 1900.
In its review, Punditfact wrote that the 50% number was based on adding the cost of government mandates to the roughly 30% that comes from taxes. "Regulation does have a cost, but the exact cost is hard to pin down," it ruled.
Mr Elder countered in a letter he wrote to Punditfact's editor, which he reprinted in his column - down to the "Larry" signature line - that plenty of conservative think-tanks, such as the American Enterprise Institute and the Heritage Foundation, would back up his claims.
He concluded the column by noting that "left-wing fact-checkers give us left-wing 'facts'".
The Tampa Bay Times made a name for itself, and won a Pulitzer Prize in the process, by rating the truthfulness of politicians. It helped spawn a whole host of similar fact-checking sites that politicians love to cite when their opponents' statements are dinged for untruthfulness, but generally the subjects of the criticism make it a policy to ignore the ratings.
They'd rather let the issued drop than turn a candidate's truthfulness into an ongoing debate.
"We're not going to let our campaign be dictated by fact-checkers," Neil Newhouse, a pollster for Republican Mitt Romney's 2012 presidential campaign, famously noted.
Pundits are a different breed, however. For one thing, they love to talk - about themselves, about politics, about media bias. And some of them, it may be shocking to learn, aren't thrilled with being taken to task in print and are willing to use their media megaphone to say so.
"I take my credibility quite seriously, and you've slammed my character and integrity," Mr Elder wrote. "Stuff like this affects one's stature and even career. You should have been more considerate and respectful."
Mr Elder certainly won't be the last commentator to fire back at Punditfact, and maybe the folks at Punditfact will enjoy the free publicity. But it just goes to show that the whole process of judging what is, by nature, a business largely based on "opinion" is going to be a tricky task.
There's an old Mark Twain quote about criticising newspapers: Never pick a fight with people who buy ink by the barrel. Punditfact is finding out that criticising commentators who measure their audience in the millions will be equally challenging.