National Journal Death Spiral Watch

Back when I was in the government, I enjoyed reading the National Journal. Its coverage of lobbying and influence peddling was the best. Its coverage of legislative process was very good. And its coverage of policy substance was OK--handicapped by he said/she said journalism and the fact that it took every Republican claim seriously, but OK.

Since then it has gone downhill. And now it has jumped the shark: IMHO, nobody should be reading the National Journal. Nobody should be paying a cent to the National Journal--you can get better, more reliable, more sophisticated information from less dishonest information brokers from other places.

Steve Benen:

The #1 most liberal senator is … Barack Obama?: Be prepared to hear about this, over and over again, for quite a while....

Sen. Barack Obama, D-Ill., was the most liberal senator in 2007, according to National Journal’s 27th annual vote ratings. The insurgent presidential candidate shifted further to the left last year in the run-up to the primaries, after ranking as the 16th- and 10th-most-liberal during his first two years in the Senate....

If all of this sounds a little familiar, it’s because in 2004, National Journal named John Kerry the most liberal senator of 2004 (John Edwards was fourth), which became one of the principal talking points of the Bush-Cheney campaign.... But before anyone takes the National Journal rankings at face value, it’s worth noting... the methodology... was misleading in 2004, and it’s equally misleading now.... Obama and Joe Biden were both considered more liberal than Russ Feingold and Bernie Sanders. This, alone, should make one wonder....

What’s more, Obama was the 16th most liberal senator in 2005, and the 10th most liberal in 2006, before racing to the front of the pack in 2007. National Journal suggests this has something to do with Obama moving to the left to curry favor with Democratic primary voters. But... Obama missed a whole lot of votes in 2007.... The rankings use an amorphous meaning of the word “liberal,” and the percentage doesn’t take missed votes into account at all (which also helps explain why Kerry nabbed the top spot four years ago)....

As Brian Beutler noted, [T]his is philistinism masquerading as social science — it’s the U.S. News College Guide of Washington politics. Journalists ought to understand that. And those of conscience ought to ignore it, or lay it bare...”

Why oh why can't we have a better press corps?

Comments

[bdbd]

"insurgent presidential candidate"??? The New Yorker cover in a few words!

[Mike]

Not knowing a thing about how the National Journal does their rankings can someone explain what it is about the methodology that is misleading?

Do they have a panel of experts assign ideological categories for each vote (yes or no) then just tally the results? If so, how does one become an accredited ideology category maker? I think that sounds more fun than what I'm doing now.

[elspi]

"NJ scored two senators running for the dem nomination as the most liberal isn't really compelling evidence that that weren't, now is it?

1. They only score there when they are running for president
(calculate the odds of that happening by chance and get back to me.)

2. Other senators are clearly more liberal.

I am aware of all your mother's internet traditions.

[Nick]

The NJ ranking has Clinton at 16th and Obama at 1st on their most liberal ranking, but they only differed on two votes:

18/S1 "Establish a Senate Office of Public Integrity to handle ethics complaints against senators. January 18. (27-71) C-1"

189/S1348 "Allow certain immigrants to stay in the United States while renewing their visas. June 6. (41-57) C-2"

[Arguably also the Iranian national guard vote, where Hillary voted Yeay and Barack did not vote but criticized Hillary's vote]

That seems like pretty shaky ground to rank someone 16th, and hence a middle of the pack democrat, and paint Barack as a left-wing extremist.
One can certainly make the argument that 18/S1 is not really a liberal position, since McCain in primary mode and Lindsey Graham both voted for it as well [or perhaps now Democrats have officially become the party of better senatorial ethics, and liberalism has been associated with more ethical conduct while conservatism is associated with corruption.]

I find that a better ranking is available that uses more information and uses senate roll call data without assuming a priori that certain positions bills are liberal or conservative.

http://www.voteview.com/sen110.htm

The methodology is a little complicated, but the results have Obama as 11th most liberal in the 110th senate and McCain as the 8th most conservative. Note that their methodology uses 388 roll calls instead of the 99 that the National Journal somehow (cherry picked with bias?) selected as the "important" votes in the Senate.

It gets even more interesting for the 109th senate, where Obama is 21st most liberal and McCain is 2nd most conservative.

I wish the mainstream press would occasionally report this study or other studies instead of constantly repeating or failing to challenge the assertion that Obama is the most liberal senator, because somehow the National Journal has a monopoly on defining the political spectrum.

As Brad and his fellow nation of whiners say:"Why oh why can't we have a better press corps?"

[Nick]

The NJ ranking has Clinton at 16th and Obama at 1st on their most liberal ranking, but they only differed on two votes:

18/S1 "Establish a Senate Office of Public Integrity to handle ethics complaints against senators. January 18. (27-71) C-1"

189/S1348 "Allow certain immigrants to stay in the United States while renewing their visas. June 6. (41-57) C-2"

[Arguably also the Iranian national guard vote, where Hillary voted Yeay and Barack did not vote but criticized Hillary's vote]

That seems like pretty shaky ground to rank someone 16th, and hence a middle of the pack democrat, and paint Barack as a left-wing extremist.
One can certainly make the argument that 18/S1 is not really a liberal position, since McCain in primary mode and Lindsey Graham both voted for it as well [or perhaps now Democrats have officially become the party of better senatorial ethics, and liberalism has been associated with more ethical conduct while conservatism is associated with corruption.]

I find that a better ranking is available that uses more information and uses senate roll call data without assuming a priori that certain positions bills are liberal or conservative.

http://www.voteview.com/sen110.htm

The methodology is a little complicated, but the results have Obama as 11th most liberal in the 110th senate and McCain as the 8th most conservative. Note that their methodology uses 388 roll calls instead of the 99 that the National Journal somehow (cherry picked with bias?) selected as the "important" votes in the Senate.

It gets even more interesting for the 109th senate, where Obama is 21st most liberal and McCain is 2nd most conservative.

I wish the mainstream press would occasionally report this study or other studies instead of constantly repeating or failing to challenge the assertion that Obama is the most liberal senator, because somehow the National Journal has a monopoly on defining the political spectrum.

As Brad and his fellow nation of whiners say:"Why oh why can't we have a better press corps?"

[Nick]

The NJ ranking has Clinton at 16th and Obama at 1st on their most liberal ranking, but they only differed on two votes:

18/S1 "Establish a Senate Office of Public Integrity to handle ethics complaints against senators. January 18. (27-71) C-1"

189/S1348 "Allow certain immigrants to stay in the United States while renewing their visas. June 6. (41-57) C-2"

[Arguably also the Iranian national guard vote, where Hillary voted Yeay and Barack did not vote but criticized Hillary's vote]

That seems like pretty shaky ground to rank someone 16th, and hence a middle of the pack democrat, and paint Barack as a left-wing extremist.
One can certainly make the argument that 18/S1 is not really a liberal position, since McCain in primary mode and Lindsey Graham both voted for it as well [or perhaps now Democrats have officially become the party of better senatorial ethics, and liberalism has been associated with more ethical conduct while conservatism is associated with corruption.]

I find that a better ranking is available that uses more information and uses senate roll call data without assuming a priori that certain positions bills are liberal or conservative.

http://www.voteview.com/sen110.htm

The methodology is a little complicated, but the results have Obama as 11th most liberal in the 110th senate and McCain as the 8th most conservative. Note that their methodology uses 388 roll calls instead of the 99 that the National Journal somehow (cherry picked with bias?) selected as the "important" votes in the Senate.

It gets even more interesting for the 109th senate, where Obama is 21st most liberal and McCain is 2nd most conservative.

I wish the mainstream press would occasionally report this study or other studies instead of constantly repeating or failing to challenge the assertion that Obama is the most liberal senator, because somehow the National Journal has a monopoly on defining the political spectrum.

As Brad and his fellow nation of whiners say:"Why oh why can't we have a better press corps?"

[Nick]

The NJ ranking has Clinton at 16th and Obama at 1st on their most liberal ranking, but they only differed on two votes:

18/S1 "Establish a Senate Office of Public Integrity to handle ethics complaints against senators. January 18. (27-71) C-1"

189/S1348 "Allow certain immigrants to stay in the United States while renewing their visas. June 6. (41-57) C-2"

[Arguably also the Iranian national guard vote, where Hillary voted Yeay and Barack did not vote but criticized Hillary's vote]

That seems like pretty shaky ground to rank someone 16th, and hence a middle of the pack democrat, and paint Barack as a left-wing extremist.
One can certainly make the argument that 18/S1 is not really a liberal position, since McCain in primary mode and Lindsey Graham both voted for it as well [or perhaps now Democrats have officially become the party of better senatorial ethics, and liberalism has been associated with more ethical conduct while conservatism is associated with corruption.]

I find that a better ranking is available that uses more information and uses senate roll call data without assuming a priori that certain positions bills are liberal or conservative.

http://www.voteview.com/sen110.htm

The methodology is a little complicated, but the results have Obama as 11th most liberal in the 110th senate and McCain as the 8th most conservative. Note that their methodology uses 388 roll calls instead of the 99 that the National Journal somehow (cherry picked with bias?) selected as the "important" votes in the Senate.

It gets even more interesting for the 109th senate, where Obama is 21st most liberal and McCain is 2nd most conservative.

I wish the mainstream press would occasionally report this study or other studies instead of constantly repeating or failing to challenge the assertion that Obama is the most liberal senator, because somehow the National Journal has a monopoly on defining the political spectrum.

As Brad and his fellow nation of whiners say:"Why oh why can't we have a better press corps?"

[Dirty Davey]

"can someone explain what it is about the methodology that is misleading?"

The methodology is to identify a certain number of specific "key votes", and calculate the percentage of Senator X's votes that are cast on the "liberal" or "conservative" side of these votes.

Among the misleading things methodologically:

Absences are ignored. Once candidates seriously start running for President, they miss a lot of votes... and are most likely to take extra effort to be present and vote when they need to support a party-line vote. So the presidential candidates end up "more liberal" than most members because they are more likely to miss the roll call votes where there is more crossing of the aisle.

McCain managed to miss enough votes that they don't even rank him... so instead of the coverage trying to realistically assess both sides of the race the NJ runs the headline OBAMA: MOST LIBERAL and doesn't even use the name McCain until the EIGHTEENTH paragraph of the story.

Of course, the selection of "key votes" is itself subject to manipulation. It's entirely possible that inclusion or exclusion of any half-dozen or so roll call votes could significantly change the rankings.

And that's the process-based argument against their methodology. The results-based argument is even stronger: that it shows a consistent pattern of ranking Democratic presidential contenders as "more liberal" than (a) Senators like Feingold and Sanders who are clearly more liberal and (b) the SAME INDIVIDUALS in years in which they are not running for President. Unless you honestly believe that the voting record shows Obama "moving to the left" of Bernie Sanders in the last year, it's clear the results are wrong.

(Though I don't think the problem is merely that they are "incorrect". The inaccuracies and biases of the method are, I'm sure, obvious to the authors and to National Journal--but they'd rather get headlines and laudatory press releases from the McCain campaign than produce a more boring and accurate map of Senate political positions that no one cares about.)

[Dirty Davey]

Thanks for the VoteView pointer, Nick! The Poole-Rosenthal (and now Lewis-Poole) estimates are as I understand it the most methodologically sound estimates of this sort of thing.

[ogmb]

Poole, Rosenthal et al. discuss Obama, Clinton, McCain and Bush here:

http://voteview.ucsd.edu/Clinton_and_Obama.htm

[Mike]

Thanks for the overview. Sounds like a classic sample bias.

Doesn't anyone read "How to Lie with Statistics" anymore in college Stats 101?

[mere mortal]

The methodology seems perfectly sound. The candidate who is most often called "liberal" by Republicans and right-leaning organizations is the most "liberal."

Obama's the guy, case closed. Hillary Clinton is a close second. She might have been #1, except they forgot to include references to "Hitlery" which came in 7th most "liberal" all by itself (mostly thanks to talk radio).

Kerry is much less "liberal" than in 2004 because they don't have to call him that anymore. Bernie Sanders doesn't even qualify, as he doesn't meet the minimum number of mentions.

Hope that helps clear it up.