What IT Teams Want Their CIOs to Know About Enterprise Big Data

Bangalore: Big Data cloud services provider Infochimps unveiled the results of a survey, which identifies the top items that IT team members would like their CIOs to understand concerning their Big Data projects. The report - CIOs & Big Data: What Your IT Team Wants You to Know-- is based on Infochimps' query of more than 300 IT department employees, 58% of whom are currently engaged in Big Data projects, and aims to identify pitfalls that implementation teams encounter, and could avoid, if top management had a more complete view.

"It isn't surprising to find that 55% of Big Data projects aren't completed and many more fail to achieve their objectives -- often those charged with implementation are the last consulted," said Jim Kaskade, CEO, Infochimps. "We created this report as a resource to give CIOs insight into the too-often overlooked views of those charged with the heavy lifting."

The survey indicates that 81% of companies include Big Data/advanced analytics projects among their top five IT priorities for 2013. According to Infochimps, CIOs who want their projects to succeed should consider the following perspectives from IT staff.

The Top Challenge: A Culture of Siloed Data Respondents said that the most significant challenge they face when working with Big Data is accessing the data siloed in various business applications across the organization (76%). In addition, inaccurate scope was listed as a top reason why IT projects in general fail (58%).
"Companies need to start with the business problem first to properly scope their projects," said Kaskade. "Too many organizations are building Big Data platforms intended to meet the entire organization's needs. Unless they understand specific use-cases first, many will find such an approach falls short."

Respondents also identified other top challenges related to their Big Data projects:
-- Staff education and understanding the platforms
-- Lack of data context and inability to connect the dots (indicating a data analysis talent gap)
-- Processing, analyzing and on-going management of the data

IT Staffers Want to Focus on the Application Layer Respondents were much more interested in focusing on the application layer than on infrastructure. "Departments will find the greatest return on investment when their time is invested honing the front-end for business agility and value, and not spent managing the back-end," said Kaskade. "New Big Data technologies also represent a huge opportunity for IT professionals to grow their talents and resumes -- departments that ignore this perspective risk high attrition in the future."

Critical Requirements for Big DataRespondents ranked as the top five most critical requirements for Big Data platforms:
-- Ability to scale
-- Ease of management
-- Flexible architecture
-- Speed to deployment
-- Security

The report also showed that, when it comes to Big Data success, Hadoop is not enough -- respondents rank real-time/near-real-time analysis and batch analysis as nearly equal in decision-making importance.

"The dirty little secret is that Hadoop is only a part of the answer," said Kaskade. "Almost every business problem requires historic data batch analytics, ad-hoc, and real-time analytics, integrated and made easy to consume by application developers. Hadoop can't do all that,"

The "One Thing" Respondents Wanted CIOs to Know: Big Data Takes Commitment When asked what single thing they wanted their CIO to know, participants offered valuable insight on subjects ranging from combining batch and real-time analysis, to overall vision. However, the theme of time remained at the forefront, indicating a desire to remind CIOs of the level of commitment required by Big Data projects:

"It takes time to interact with data"... "the time it takes to normalize data so that it can be analyzed properly"... "it has inertia, and structural changes don't happen quickly"... "time to put in place"... "time to develop"... "time and effort required"... "this will take more time not more people"... "nothing is fast"