Political polarization of the U.S. Senate: Is the data fooling us?

Earlier this month, The Economist, Yahoo! News, and several other respectable news outlets ran articles talking about some great network visualizations apparently showing the “political polarization of the U.S. Senate.” There, they argued that these visualizations show how the Senate has evolved from a fairly cohesive unit in 1989 into a dysfunctional group divided along party lines in 2013. If you look at the visualizations of voting behavior in the U.S. Senate below, you’d probably agree with their conclusions.

Visualization c/o

The supposed “political polarization” of the U.S. Senate
Visualization c/o Renzo Lucioni

I was a little skeptical. There were a couple issues with how the visualizations were presented:

  1. The divide over time wasn’t as clear as the news articles made it appear, which made me suspect that they were cherry picking.
  2. All connections between Senators who voted on less than 100 bills together were arbitrarily removed, which could have produced the sudden “polarization” effect we were seeing.

Renzo was kind enough to point me to his script that collected all the data for these visualizations, so it was easy enough for me to collect the data myself and run my own analyses. I’ve provided the data online free to download on figshare.

Were the news outlets cherry picking to prove their point?

To address the first concern, I measured the modularity of the networks over time. This modularity score basically gives us a measure of how much the Senators are divided up into disconnected groups, or political parties in this case. In the graph below, I called this measure “divisiveness.”

Divisiveness of the U.S. Senate, quantified

Divisiveness of the U.S. Senate, quantified
The x-axis is time, and the y-axis is the modularity score

It’s quite amusing to see that the first two network visualizations that The Economist showed — that were meant to show the Senate as a fairly united unit — were at the two points of lowest divisiveness in the entire time period (1989 and 2002). Similarly, the final network visualization that they showed — that was meant to show a divided Senate — was at a point when divisiveness was second highest (2013).

Coincidence? For the choice of the 1989 and 2002 visualizations, I don’t think so.

It’s fairly clear that the news outlets were cherry picking the visualizations to prove their point.

Does the data really show a divided Senate?

My second concern was that, by removing all connections between Senators who voted on fewer than 100 bills together, Renzo could have produced the sudden appearance of “divisiveness” in the Senate when the Senate has always been divided. If you look at the “divisiveness, quantified” graph above, the Senate has always been fairly divided since the early 1990’s. If that’s the case, why do Renzo’s visualizations look fairly cohesive early on, but suddenly divided around 2013?

This interactive visualization on a Yahoo! News article illustrates my concern best: If you cut many connections, the Senators form into clusters based on political affiliation. If you don’t cut any connections, then the divide is much less clear.

Cutting many connections produces the appearance of a divided Senate

Cutting many connections produces a divided Senate in 2013
Screen shot taken from this visualization

Cutting fewer connections shows a much less divided Senate

Cutting fewer connections produces a much less divided Senate in 2013
Screen shot taken from this visualization

Moving the cutoff threshold around easily changes whether it looks like the Senate is divided or not. In fact, if you play around with that interactive visualization enough, you’ll see that you can eventually produce a politically divided Senate for every year by removing enough connections. Or, if you want, you can produce a politically united Senate by leaving more connections.

So what does this tell us? Politicians have always been more likely to vote along party lines, even in 1989. Perhaps they’ve been slightly more inclined to do so this year than in 1989, but it’s not as extreme as Renzo’s visualizations suggest. The political parties only appear fairly cohesive in 1989 and completely divided in 2013 because of an arbitrary cutoff, and we could tell a completely different story if we chose a different arbitrary cutoff.

Always be skeptical of data visualizations

The point of this post wasn’t to rail against Renzo nor any of the news outlets that hyped his network visualizations. Instead, I hope you found this to be a cautionary tale to always question how data visualizations are made. Data visualizations can often be manipulated to demonstrate any point the author pleases, and it’s easy to accept what the visualization claims when it agrees with our intuitions.

Mark Twain once famously wrote:

“There are three kinds of lies: lies, damned lies, and statistics.”

I’d like to expand on his quote by adding one more kind of lie:

“There are four kinds of lies: lies, damned lies, statistics, and data visualizations.”

Dr. Randy Olson is the Chief Data Scientist at FOXO Bioscience, where he is bringing advanced data science and machine learning technology to the life insurance industry.

Tagged with: , , ,
11 comments on “Political polarization of the U.S. Senate: Is the data fooling us?
  1. Justin says:

    Really nicely done. I would still like to see an evolution of your no-cuts-made graph over time. As a qualitative response, things seem very very different to me than they did 10 or 20 years ago. Quantitatively, there must be some difference.

    • Randy Olson says:

      Thank you Justin! You can see the past 10 years in the interactive visualization I linked: http://news.yahoo.com/senate-social-network-diagram-mcconnell-mean-girls-000513361.html. Go to any year, move the slider down to the single digits (< 10%), and they all look the same regardless of the year. I suspect it would look the same pre-2001 as well. I don't show the quantitative measure of divisiveness on the raw data here due to space limitations, but it follows the same general trend of slightly higher divisiveness over time. It starts around 0.05 at 1989, and peaks around 0.19 in 2013. There isn't that sharp transition into an entirely politically divided Senate in the raw data, as the original visualizations suggest.

  2. Can’t say I agree.

    The process by which the graph becomes more and more clustered as you increase the cutoff is actually the whole point.

    The cutoff threshold is a proxy for agreement. So, the higher the cutoff, the more agreement there is (between nodes). Having a low cutoff tells us very little about bipartisanship because that’s not how we naturally define it. We don’t care if politicians from different parties EVER agree, but how OFTEN they agree.

    Granted there is no hard and fast rule about what threshold moves us from disagreement to agreement space. But if you look at the data both across the time and threshold dimensions, it’s hard to argue that it’s not painting a picture of less bipartisanship.

    • Randy Olson says:

      To clarify, I never said there wasn’t a trend of increasing partisanship. The line graph I presented in this post clearly shows that. My point is that the visualizations with the arbitrary cutoff told a different story, that the news outlets then picked up on: The Senate was fairly united in 1989, and split in 2013. This clearly isn’t the case. The Senate has always had a noticeable level of partisanship that could have been revealed with a different cutoff threshold.

      What makes for a better cutoff threshold? As you said, it’s hard to decide. Jake had a good suggestion in another comment: http://www.randalolson.com/2013/12/21/political-polarization-of-the-senate-is-the-data-fooling-us/#comment-312

  3. Jake says:

    Doesn’t each year include a large number of procedural and ceremonial votes? Seems like there would be good reason to set a threshold of votes meriting a connection that was high enough to screen out that noise.

    • Randy Olson says:

      Good thought Jake! I thought the same: It should be easy enough to look at the data and find bills that everyone voted yes/no on, and cut those out via the threshold. That’d probably be a more fair cutoff threshold to go with.

  4. ewortesting says:

    This is a bit inaccurate of an analysis too. The Yahoo visualization is the nicest since you can interact and actually see what’s really there: The divide *has* increased. It may not be as black and white as the news presents it, but as you say ‘you can *eventually* produce…’. The amount of divide does seem to be increasing.

    • Randy Olson says:

      Yep, that’s the point I was making! 🙂

      The original author of the visualizations should have either:

      (1) established a less arbitrary cutoff threshold, such as one based on statistics (e.g., this paper) or some reasonable heuristic (e.g., as suggested by Jake), or

      (2) explored the effect of changing the threshold and made sure that it doesn’t change the story that the visualizations told, as I did here.

  5. Bradly Alicea says:

    Great re-analysis. Measuring the modularity can tell us a lot about the overall trend in partisan segregation over time, particularly when there is heterogeneity within single parties (e.g. so-called internal “civil wars” and changing coalitions). Although I suspect what defines the difference between past eras and now is that so-called “weak ties” (indirect access to the various subnetworks) between the Republicans and Democrats have collapsed. Haven’t had a chance to check the dataset on this…..

  6. Hello folks, why do we divide ourselves, the Republican – Democrat blame game religion is used against us, in fact it is needed to keep us fighting while the powers that be operate with impunity. The people who are really running the show need to give us the illusion that our little vote actually has some kind of political capital at the table of the elite. It doesn’t, but once a side is chosen it gives us the impetus to vent, defend and fight amongst each other which keeps us out of the way and relying on the presstitues for the news that frame the latest False Flag. They have us divided, distracted and in fear. It is the good cop bad cop they play to pacify the sheeple. Hey, look over there it’s a shiny object!

2 Pings/Trackbacks for "Political polarization of the U.S. Senate: Is the data fooling us?"
  1. […] time and time again we have seen that data visualizations can easily be manipulated to lie. By misrepresenting, altering, or faking the data they visualize, data scientists can twist public […]

  2. […] kunnen echter eenvoudig gemanipuleerd worden en blijken vaak niet zo waarheidsgetrouw te zijn als misschien lijkt. Door data foutief weer te […]