Monday, October 3, 2011

Big Data

I was recently quoted in an article by David Rosenbaum of CFO Magazine on the use of data at American Airlines in the 1980’s and at FedEx over the last decade http://www3.cfo.com/article/2011/8/analytics_that-new-big-data-magic . As is the nature of these articles, there was some information left out that would have made the article even more useful.


One reader, Timo Elliott, wrote me to ask some clarifying questions and followed up in his blog http://timoelliott.com/blog/2011/09/relying-on-data-can-lead-to-the-wrong-decisions-says-cfo-com.html .

I want to go into a little more detail on both the AA situation and then clarify a point on the FedEx aspects in the article.

The technical situation at AA in about 1986 was that a Teradata DBC 1012 was a very small machine so all the data we could load into it was the flight data of the AA Gold Card members. At the time gold cards were given to something like the top 2% of the flier in a market.

Tom Plaskett was the senior VP of Marketing and he did ask us to see how many AA Gold Card members were flying British Air (BA) from Dallas to London Heathrow. As I explained to Timo, at the time you could fly on BA from Dallas and get AA frequent flier miles. So, on a head-to-head basis, the best AA customers preferred BA and did so overwhelmingly.

It isn’t worth the time to review the airline situation in the 1980’s. It was turbulent with lots of airlines failing and new ones starting up. It was combat.

AA had a plan to compete and part of that plan was to improve the quality of the product from end-to-end. The data on the Gold Card members was just one input but seemed to support the contention that customers would pay more for quality.

AA was wrong. It spent a $billion to improve everything from on-time arrivals to how the wine was served in first class. Didn’t make any difference. While the data clearly answered the question asked, it didn’t give the solution even through it appeared to—make AA flights as good as BA flights.

I could go into details about why the quality plan didn’t work but that is beside the point. The point is that the data can tell you what is going on but it isn’t predictive. To develop a solution you have to do more, maybe a lot more, analysis.

I want to correct something in David Rosebaum’s article about FedEx. In his article he says that when a newly signed customer doesn’t start shipping when expected FedEx pings the customer’s Blackberry wanting to know why. Not exactly correct. It is the salesman’s Blackberry that gets pinged. It is the salesman’s job to find out if there is a problem and get the customer shipping.