HP: Big data is creating jobs, but there is no one to fill them

Unstructured data explosion is a problem for all - including vendors, says former Autonomy executive. Information analysts apply here...

Big data is creating new roles, not destroying them, according to a senior HP executive. The problem is a dearth of information analysts - even for vendors themselves.

According to Nicole Eagan, HP senior vice-president of information management, the explosion in unstructured data is a real challenge for enterprises. There are not enough skilled people to turn that data into business information.

Speaking in London, she said: "With 62 per cent year-on-year [growth in] unstructured data, you do not have enough people to look at the data. We don't have people to do analytics."

Eagan made the comments in response to a question about technologies, such as HP-owned Autonomy's, replacing human beings.

"We need people to process and act on human information by using analytics from these big data sets. There's so much human information and there is no way to process it all," she said.

Eagan told Computing that this is a challenge for vendors themselves, and that the analytical skill set that Autonomy is looking for is different to that of the past.

"The hardest people to acquire are information analysts, especially when it comes to dealing with this new probabilistic technology. In the past, most of the IT industry was used to using programming languages and writing code.

"Now, the technology we deal with is a probabilistic battle, and therefore it takes a different mindset and different types of analytic skills for analysts to be able to apply a probabilistic model," she said.

IT staff's fears that they risk being replaced by automated systems that can analyse big data are unfounded, according to Fernando Lucini, chief architect at Autonomy,

He argued that big data is creating opportunities - and that any IT specialists who are currently unemployed should have developed the requisite skills to exploit big data.

"I don't think all of the people currently unemployed were looking at big data [before losing their jobs]. Big data is creating new categories [of role]; it is not creating unemployment. Companies still require somebody to take the outcome of the analysis and interpret it," he said.

However, Lucini went on to suggest that if technologies to handle big data did cause unemployment, then that should not change a company's view on developing the technology.

"A company does not stifle innovation just because it doesn't want people to lose their jobs," he argued.

Susan Feldman, research vice-president of search and discovery technologies at research firm IDC, agreed. "The challenges for IT organisations are greater than the people problem," she said.

Feldman said that IT professionals should develop analytical skills as a priority.

"It is not the technology, it is how people use it that determines how many jobs are lost or made," she said. "Employees that are within this shift are stuck because re-educating themselves and getting a different kind of job is tough. Flexible people are more employable."