Arik Johnson

Data streams are growing at a rate of 60% a year, and data use is reaching 35.2 zettabytes globally. This year, a billion terabytes of data will flow around the world, according to IDC. Given the need for tools that collect, cleanse and store all this stuff, that increase puts CIOs into prime position to become the keepers of business data. As it turns out, however, building better data analysis skills across the employee base is what's going to make the Big Data engine churn, experts insist.

More on Big Data

Big Data: Probing global warming with object database engine

Large data sets offer insights, require a tiered storage strategy

Enterprise business intelligence and Big Data quiz for CIOs

At least, that's the take of The Corporate Executive Board Company (CEB), which recently developed a test for measuring corporate employees' "analytical maturity." "The rate-limiter that stops us from making use of reams of data is us," said Shvetank Shah, executive director at Washington-based CEB's Information Technology Group. 

Shah is convinced that developing analytical skills among a broader base of employees is critical to companies' ability to make more effective use of data. "No matter how good a report, or how well we massage the data, or how much data-computing power we have, at the end of the day, decisions need to be made by humans; and we're not equipped to make those [data analysis] decisions very well," Shah said.

How ill-equipped for data analysis are we? The CEB surveyed 5,000 employees at 22 global companies and found a "serious data-analysis insight deficit," Shah said. Of those surveyed, 43% were deemed unquestioning empiricists -- they trusted data and statistics over judgment, and valued the consensus. Another 19% were visceral decision makers -- they seldom trusted data analysis and made decisions unilaterally.

What companies need is employees who are informed skeptics, Shah explained. Such employees (38% of those surveyed) apply judgment to data analysis, listen to others and are comfortable with dissent.

The informed skeptic's data analysis skills

Companies whose employees had the highest level of data analysis skills, or insight IQ, as CEB calls it, were those that:

• Explain the limitations of data to employees.
• Develop an analytical training curriculum.
• Hire quantitative experts who can coach.
• Formalize the decision process.

Here is a sampling of the questions payment processing company First Data Corp. uses to test and sharpen its employees' critical thinking skills as part of a program to develop more informed skeptics:

• Clarity: Could you illustrate what you mean? Could you give examples?
• Accuracy: How could you find out if that is true?
• Precision: Could you be more specific? Could you give more details?
• Relevance: How does the data relate to the problem?
• Depth: What difficulties arise that we might need to deal with?
• Breadth: Do we need to look at this problem from another perspective?
• Logic: Does the data make sense together?
• Significance: What is the most important problem to consider?
• Fairness: Do you have biases that will affect the outcome?

On the CEB website is an analytical skills test that will help employees evaluate whether they are an informed skeptic, visceral decision maker or unquestioning empiricist.

Embedded data analysis tools

Not everyone believes the human factor will necessarily make or break a company's ability to capitalize on its data. Analytics increasingly are being embedded in applications and business processes, analyst Kurt Schlegel points out. That might lessen the need to develop employees' analytical skills, because the analysis is simply built in and done for the user.

No matter how good a report, or how well we massage the data, or how much data-computing power we have, at the end of the day, decisions need to be made by humans.

Shvetank Shah, executive director, CEB Information Technology Group

"Look at the recommendation engines on the Web as an early example of this," said Schlegel, research vice president at Stamford, Conn.-based Gartner's analytics and business intelligence group. "Soon, this technology that combines analytics with customizable business rules will lead to more personalized business processes across a range of fields way beyond e-commerce."

Giving data analysis tools to employees, however, might not be enough, no matter how intuitive they are. A performance management culture also needs to be developed, said Boris Evelson, vice president and principal analyst at Forrester Research Inc. in Cambridge, Mass.

"You have to go department by department, business unit by business unit, and have a very clear strategy for that department and unit," Evelson said. "If you have a clear strategy, you know what goals and objectives you need to achieve and what questions need to be asked to achieve those goals. Then you can decide what metrics you need to be monitoring to answer those questions and meet those goals, and what applications will help you get at that data."

The CIO's role in analytics

CIOs are the ones who can make sense of all those streams of raw data. "That's a very complex process that the CIO can own, finding the raw data, extracting it, cleansing it, aggregating it, mapping it, securing it and storing it," Evelson said. "Once the data preparation process is done, I think the CIO needs to step away and say to the business, 'Here's all the data you'll ever need. It's secure. Do with it what you want, and if something is missing or inaccurate, let us know.'"

The most effective CIOs will be the ones who are not just data gatekeepers. "They need to be business savvy," said Cindi Howson, founder of BIScorecard, a Sparta, N.J.-based online information service.

After all, technology does not take away the need for human insight and action. "Technology can help us sift through the data and discover the patterns, but it is still up to people to make the decision and then act upon that decision," Howson said.