Big data part 3: case studies

University of Oxford

Accurate analytical and reporting tools are essential in providing different users with the intelligence they need. Andy Cotgreave is senior data analyst at the University of Oxford, where staff need to quickly extract large volumes of information concerning students, examination results and course modules from a variety of different sources, before compiling it in reports that are easy to access and understand.

“We installed a new student system around 2004, and in the process did not think about how to get the data out: the ability to do any reporting on it was effectively chopped off,” says Cotgreave. “We had big unsupported tools like Oracle Discovery but that could only be as up-to-date as the IT department made it and it was very hard to get the data required.”

To solve the problem, the university has been using Tableau Software’s Server and Public visualisation applications. These applications allow up to 700 college administrators, academics, course directors and senior management to access reports produced by the Server module. The software also fronts an online portal where anyone can log in to extract statistics on student numbers, fees and careers information.

“We can take data from anywhere – student data from student systems, whether Excel, Access or CSV files – and point Tableau towards them and do analytics really quicky,” says Cotgreave. “In the past we had to import everything into Excel.”

Cotgreave publishes a mixture of reports, some once a year like examination registration and results, but also handles about 10 to 15 ad-hoc requests a week. Tableau’s simplicity also allows staff to run their own analytics, searches and reports without asking IT staff to produce them in their behalf.

“We realised that we were still constrained to the data reported by the IT team – it was hard to tweak it and there was a six-month waiting list – so we realised we needed a system where we could get the data itself by directly connecting to the data tables, using SQL query tools, where we could get data in any shape or form and open the flood gates internally, then publish reports across the university very quickly,” says Cotgreave. “Because we control the data and the reports, the response times are immensely fast, whereas the IT guys are left to focus on infrastructure and security.”

Tableau’s innovative data visualisation capabilities allow it to present information that was previously hidden, and also give the university fresh scope to explore data in ways that were not possible before.

“Before, the culture was to have a table of every single number, so the information was thorough, but the story of the data was completely hidden in an Excel wizard-based approach so people found that it was not quite what they were looking for but could not be bothered to go back and search for it again,” says Cotgreave.

“The most well-received visualisation we did was a big data survey highlighting the differences among potential students. We did not know what the end chart would be, but found we could play with it until we got to a Eureka moment where it was exactly what we needed.”

Honda

Elsewhere, automotive giant Honda has started using SAS Customer Intelligence analytics software to help predict the future success or failure of the motorcycles intended for production.

“We have collected a large amount of research data over the years that isn’t just quantitative (such as sales performance and revenues or technical-motor patterns) but also qualitative (such as judgments, values and preferences expressed by customers),” said Daniele Lucchesi, market and product research manager at Honda R&D Europe.

Lucchesi uses SAS to integrate all of Honda’s data into a single data warehouse prior to analysis, then produces regular, automated reports that attempt to identify correlations and buying patterns which can influence motorbike design.

Deutsche Postbank London

Rather than interpreting large amounts of data, the requirement at Deutsche Postbank involves high-value financial transactions that need to be regularly extracted and analysed quickly. Clarel Sookun, head of IT at the bank’s London office, is in the process of upgrading its systems to help improve the quality and accuracy of the bank’s business reporting to provide it with a better idea of the risks associated with lending money to specific corporate customers or delivering new financial services within its commercial real estate (CRE) division.

Though the actual volume of information is small – only 400-500GB in total – it is spread across multiple sources and changes on a daily basis, while staff need to get hold of the latest information and be certain it is 100 per cent accurate to make lending decisions quickly.

“Reporting has always been a problem because the bank extracts information in various forms from various sources – we used to work with an external organisation but unfortunately they went into liquidation and we could not carry on with that [same system],” says Sookun.

The bank recently switched to using IBM’s Smart Analytics, which allows it to pull information from DB2, iSeries mainframes and SQL databases, RAW file formats and in-house systems developed with Microsoft Access into a central data warehouse every night, ready for users to run reports first thing in the morning.

“Some data comes from spreadsheets, though we are trying to move away from that,” says Sookun. “With the old system repositories, someone had to go and write a program that extracted the data, or export it into spreadsheets before it could be manipulated into a report you wanted, and that took a long time.”

Reports now take a matter of minutes, rather than hours, even though the IT department still has to process them.

“At the moment, the users have limited access and they tend to come to the IT department – it only takes 20 minutes or half an hour to produce a report – but we will give users the ability to produce their own reports later,” says Sookun.

“There is quite a lot of consolidation and data cleaning involved, which has helped a lot, though the information we extract is done selectively. We do not grab everything – otherwise, we end up with huge databases with lots of inaccurate data. We make sure we can reconcile every piece of information on the database, verify, it and check it against other reports.”