Excellent data comprehension video

A colleague forwarded a link to me showing a video explaining Mariano Rivera's magical pitch: http://www.nytimes.com/interactive/2010/06/29/magazine/rivera-pitches.html
I hope he's able to keep pitching after today's knee injury.

Investment Portfolio Rankings De-cluttered

At my work we track and rank various statistics across more than 200 investment portfolios. One of the managers with 2 portfolios, used to be ranked in the top quartile for a certain statistic, but I noticed recently that they are now only top-half. To better understand what happened, I grabbed all of the portfolio rankings over time and plotted them in a single line-chart - If you are familiar with soccer, then think League-chart or rankings-chart. With so many lines, I basically ended up with a noisy plot that made little sense. What I realized though is that by selecting Excel's gray color palette, I could blend together all of the uninteresting portfolios and then thicken and color the two in which I was interested. I like how all of the background noise becomes something of a wall paper that you don't see.

There was one other trick. I needed to raise the two highlighted data series to the top of all of the other lines, and that was slow to do using the 'data series' selection box arrows. I ended up rearranging my underlying data columns of data to get the same effect. The rightmost columns of data in Excel were the top-placed lines on the graph.

Takeaways:

1) I didn't actually use this chart in my eventual writeup. I simply used to to figure out that something happened in Feb-Mar 2011, and so I dug into investor reports from these two dates for these funds and wrote down what happened. In that sense it was an exploratory graphic versus a story-telling one. The actually story of what happened in the portfolio is more important than the rankings.

2) I don't think this could ever end up in marketing materials - a league rankings type of chart is pretty difficult to explain and is an extra step removed from the underlying data. There are much simpler ways to show that something happened in Feb-Mar 2011.

3) That said, I do think the gray palette combined with highlighting certain data series is a great way to de-clutter a noisy graph, especially since it only took 2-3 minutes in Excel. I'd love to find a more practical use for it in the future.

Visualization in the Kitchen

We recently renovated our kitchen. We thought it would be simple because there weren't any existing cabinets and we weren't moving any walls, but boy were we wrong! One thing that did help out a lot though was our data visualization tool, Google Sketchup. Sketchup is a beautiful and free-to-use tool and after spending a solid weekend with it, maybe 20hrs, I had a great model that my wife and I used to tweak our design. We uncovered many flaws in our initial ideas that just wouldn't look good or wouldn't even work. Google Sketchup was invaluable and saved us many troubles and hassles. It also helped us to negotiate with and explain our design to contractors, prior to them bidding.

Here are a few renderings of our final model:

Once we created this model, ordering cabinets was a snap because most of the details and dimensions were already figured out. We worked with somebody from Lowe's who offered some great suggestions; for instance he suggested swapping the fridge and the pantry so that the fridge would be easy to load and unload next to the counter. He also knew much more about cabinet door clearances and spacing than I.

It wasn't so easy to take pictures of the final work because you can't do a cut-away or an overhead shot, but here are some photos. I'll let you decide for yourself how close we were to the model, but we are very happy with the results:

I like pie (charts)!

The field of data visualization is an means to an end. Its existence and recent rise in popularity can be attributed three things:

1. Richer data in the workplace
2. Better and more accessible tools for storing, transforming, and understanding data
3. Workers have matured and developed better technical and analytical skills for understanding data analysis

If you read blogs, there seems to be two broad categories of data visualizers: the first categories are artists and graphic designers who create visualizations as a form of expression, and the second are journalists who use visualizations to tell a story with data. I enjoy and appreciate the creativity that these two groups and their work has been an inspiration to me, but I also feel that these groups get lost in the means of data visualization and often the points they are making are either not important or just not clear.

There are a much larger and less blogged-about world of data visualization that occurs within science and business. In these fields, people need to understand their data and communicate about it. This larger area of data comprehension does not always involve a pretty visualization and at times does not require any visualization at all. In fact, spending too much time creating a visualization would be waste once you have done the work to discover or present your idea clearly.

This blog is about the practice of data comprehension. My goal here is to share with and to learn from practical people who work with data for a living. We are all artists and we all have a story to tell with our data, but we are limited by the constraints of our daily jobs.

-scm