The importance of agile data architecture in financial services

clock • 3 min read
The importance of agile data architecture in financial services

The importance of agile data architecture in financial services

The case for modernising data infrastructure to keep up with the fintechs

The move by traditional financial organisations to offer a modern interface to customers has been progressing for a number of years via digital transformation, new apps and services and back office modernisation.

They have been obliged to modernise because of regulations such as PSD2, consumer expectations, and the rising tide of fintech startups which threaten their market share.

The raw material for this transformation is data. Financial organisations are major employers of data scientists and use the models they create to optimise their services, create new products, assess risk and reduce fraud. They also have a great deal of data at their disposal, and make more use than other sectors of streaming technologies.

In view of the vast volumes of data flowing through financial organisations, the need for real-time information, as well as the many regulations with which they must comply, the infrastructure that channels the data to where it's needed is of paramount importance.

Unsurprisingly, with their inevitable technical debt, the quality of the data infrastructure that exists in venerable financial organisations, is a mixed bag, according to a recent Computing survey of 150 IT leaders, with most expressing a middling degree of confidence.

Linked to confidence is the ability to make the best possible use of the data at their disposal. Again, while some financial organisations are well advanced in this regard, others are not. Fifty-three per cent make use of less than half of the available data, with a fifth utilising less than 30 per cent.

Blockers to the sort of data agility that most desire include integration issues, concerns about security, and the sheer complexity of their infrastructure, again, not surprising in a traditional, highly regulated sector.

Looking more closely at integration issues, financial organisations were much more likely to mention poor data quality than other sectors. A whopping 74 per cent of financial services companies represented here raised this as a problem, compared to 49 per cent of companies as a whole. Data quality is an ongoing bugbear for finance precisely because it is such a heavily regulated sector, and much resource is expended in cleaning and validating data on arrival.

The issue of data quality is closely related to the second and third most frequently raised issues, namely data platforms and formatting. There can also be issues with third-party data, as this respondent said.

"Data quality is the main challenge as we rely on master data sent by external partners - we have no control over the quality of this data, and we regularly see issues where mandatory data elements are missing or incorrectly formatted. These cause issues in day-to-day operational situations and subsequently in the data warehouse."

Integration issues also extended to the cloud, where IT leaders in finance felt they were behind the curve.

Taken together, these responses make a strong case for the automation of ETL (and/or STL) to clean, deduplicate and transform data into the necessary format, preferably - in view of the ongoing skills shortage - one with a simple UI and lo-code functionality. These tools should be designed for the streaming era, rather than the batch processing of yesterday.

Data-driven strategies in financial services require agile data architectures and trusted data, and ingestion, integration and preparation platforms need to align with those goals.

For the UK financial sector, the alternative is to fall further behind in our increasingly data-driven recovery, and risk eventually being outrun altogether.

Download the Computing research white paper, sponsored by Qlik: How Agile Data Architectures Will Power Recovery in Financial Services

You may also like
Roku reports cyber breach impacting 576,000 accounts


Compromised data was used to make unauthorised purchases in fewer than 400 cases

clock 15 April 2024 • 2 min read
How do you solve a problem like [REDACTED]?

Big Data and Analytics

Developing complex digital solutions to reach the proof-of-concept stage is even more of a challenge when you don’t have access to the sensitive data you need.

clock 06 November 2023 • 4 min read
Flying high in data science


An interview with Michelle Conway of Lloyds Banking Group

clock 11 October 2023 • 6 min read

Sign up to our newsletter

The best news, stories, features and photos from the day in one perfectly formed email.

More on Finance

Raspberry Pi announces plans to go public on London Stock Exchange

Raspberry Pi announces plans to go public on London Stock Exchange

The news comes as a welcome relief for the LSE

clock 16 May 2024 • 2 min read
Cybersecurity Festival 2024: Four ways to cut your cyber insurance premiums

Cybersecurity Festival 2024: Four ways to cut your cyber insurance premiums

Certifications mean nothing without action

Tom Allen
clock 08 May 2024 • 4 min read
Built world climate tech stands out against VC slump

Built world climate tech stands out against VC slump

The UK is in a strong position for global investment

Penny Horwood
clock 06 December 2023 • 3 min read