Middle East CIOs must learn more about Hadoop, says Teradata CTO

CIOs in the Middle East need to start to learn what Apache Hadoop is in order to understand the true value of big data, according to Stephen Brobst, CTO, Teradata.

Brobst was speaking to CNME at Teradata Universe in Copenhagen, where the company today became the first company to offer business analysts streamlined and self-service access to Hadoop.

“The average CIO may or may not know what Hadoop, and certainly doesn’t know what SQL-H, means to them,” Brobst said. “Hadoop is something they ought to understand because open source technology allows you to change the economics on gathering very large data.”

Big data is gradually transitioning as a buzz term to a top priority for senior IT executives across the Middle East, who are looking to extract value from their data.

Apache Hadoop is an open-source framework that allows organisations to process large amounts of data, regardless of its structure, at a low cost, and has largely been attributed to driving the growth of the big data industry.

However, Hadoop has presented the industry with challenges related to security, standards, and most significantly, a lack of talent in data science.

“I do a fair amount of work in the Middle East and there are definitely countries that are more sophisticated than others – specifically in Saudi, Egypt, and to a lesser extent, the UAE,” Brobst said. “They are starting to explore the opportunities for using big data and open source technologies to change the economics of dealing with this big data.

“By big data I don’t only mean high volume data, I mean diversity of data from non-traditional sources like web log and OSS data. From a CIO perspective it’s about changing the economics to allow them to get access order-of-magnitude more data without having to pay order-of-magnitude more cost to their bottom line.”

Teredata’s new solution – Teradata Enterprise Access for Hadoop – allows organisations to reach directly into Hadoop to find new value from data analysis and enable quicker and smarter strategic decisions, the company said.

It enables data scientists to escape application development and work on high-value activities. In addition, data scientists can gain a better understanding of the data lineage to determine its reliability and accuracy.

“The issue and opportunity is that if I only store a bunch of data very cheaply, who cares because I’ve got to create value from it. So the idea related to the platform announcements is you want to be able to get access to that data in a way that does not require you to have an army of PHDs in computer science.”

According to Brobst, a lot of the big data technologies were developed mainly by dot com companies in Silicon Valley, which are rich in Stamford PHDs and Java programmers.

“The average telco or bank in the Middle East is not rich with Java programmers or PHDs from Stamford University,” he said. “So you need a platform that allows a data scientist who is technology competent and savvy, but not a computer scientist, to get value from the data.

“The bottom line for a CIO is to give value to the business using a set of skill sets which we have at order-of-magnitude lower price-per-terabyte for accessing that. You leverage the open source technology, but add value-add on top of it that allows the data scientist to be successful, because the skill sets are not as available in the Middle East as they are in Silicon Valley.

“We just have to face that reality, and that would be true in almost anywhere compared to Silicon Valley, quite frankly. And even in Silicon Valley the skill sets are scare, much less outside of Silicon Valley.”