Should You Use A Data Driven Approach

The inspiration for innovation takes all forms. For some its music. For others its art. And for others its data.

When I was CTO, Mark Hurd, the CEO at HP at the time, had a quote that was ingrained into everything the executive team did.

“If you stare at the numbers long enough, they will eventually confess.” Mark Hurd

The expectation was that as an executive you knew “your numbers”. It was not unusual to have Mark stop me in the hall and ask about the R&D investment levels last quarter for the top three competitors, customer net promoter scores for our top 5 products or the reverse supply chain levels from retail returns.

While Mark’s focus on the numbers was well-meaning, I always felt that it caused blind spots when it came to understanding the shift, changes, and unspoken needs and wants of our customers. It looked at numbers as single elements to be managed individually. It also had the built-in assumption that the numbers were fact and that they never misled.

Years later, I came across this story that caused me to reflect back on these times at HP.

Misleading Innovation Decisions

During WWII, the Navy tried to determine where they needed to armor their aircraft to ensure they and their crew came back home. They started tracking each and every bullet hole from each plane in the navy. With this data, they ran an analysis to see if there were any trends of where planes had been shot up.

Based on the analysis, the conclusion was that they needed to increase the armor on the wingtips, on the top of the central body, and around the elevators. That’s where the data told them their planes were getting shot up.Abraham Wald, a statistician, disagreed. He thought they should put more armor in the nose area, engines, and the underside of the fuselage.

Everyone immediately thought his proposal was crazy. That’s not where the planes were getting shot.Except - Mr. Wald realized what the others didn’t.

What the Navy thought it had done was analyze where aircraft were suffering the most damage. What they had actually done was analyze where aircraft could suffer the most damage and still make it back.

What about the places where the planes in their analysis were not shot? Put simply, planes that had been shot there crashed. They weren’t looking at the whole sample set, they only looked at the planes, and crews that survived.

Did The Data Lie or Just Mislead?

The data didn’t lie. The planes did get shot in the locations identified during the analysis. The data, however, did mislead. It was only part of the entire data set that should have been looked at.

While data can be incredibly helpful when developing ideas that will become future innovations, we need to apply human insight and skepticism. Throwing in your gut feel may also be a good idea.

If something seems incredibly obvious, that begs the question as to why and what is missing. Rarely are things that cut and dry. That obvious.

Go beyond the obvious and use your curiosity to ask that next question so that you can dig deeper and uncover some insight that others are not seeing.

Be careful of assumptions. Be careful of using past experience or even what we think we see and then filling in the missing data.

Impact on HP

So what happened with the Mark Hurd approach at HP? With the emphasis on your numbers being compared to your competitors, it became clear that if your numbers were not “better” than theirs’ then you weren’t running your part of the business appropriately.

The result was some bad business decisions such as cutting HP’s R&D spend to match the R&D spend of our Asia Pacific based competitors. I always found it interesting that the focus was always on cutting. Why wasn’t the decision made to increase the R&D spend to match that of Apple?

That is a story is for another time.

While I pushed back hard on this approach and specifically what was being done to R&D spend, my one regret was not pushing back even harder or finding a way to convince Mark and others on the folly of the approach. I didn’t find a way to play the “Abraham Wald” role at HP.

One key lesson that I did learn from this experience was that the context of the information you are using to make innovation decisions is just as important as the data.

Killer Question

How could you challenge yourself and your team to take the “Abraham Wald” approach with the aircraft analysis? How can you go beyond the obvious and uncover an insight that is not obvious?