Big data part 4: analyst's view

As the technology stack for big data processing matures, vendor offerings and the availability of utility computing will lower risk, improving the business case for investment. Position your company to gain a competitive advantage as “the pack” races to catch up. This requires your business to shift its thinking about business data from a cost to an asset. It also requires IT to avoid the temptation to view big data as a point solution in its own silo. Enterprise architects should lead their firm’s thinking by:

• Assisting the business in opportunity recognition. Use a business capability map to foster dialogue about opportunities to extract value from sets of data thought too large and cumbersome to handle. Develop powerful stories using business scenarios, then drill through to affected processes and metrics. Business scenario analysis creates a narrative of people, process and technology changes required to get from where the business is to where it wants to be.

• Maintaining a holistic approach to information architecture (IA). Update your IA strategy considering the special characteristics of candidate big data sets. Define metadata to understand key attributes required for analysis and develop a sourcing strategy to answer important questions about where the data will come from (organic growth or acquisition), how fast it must be captured, how good it is, and where it will be maintained. Ensure data governance processes are adequate for the volume, velocity and content of proposed big data sets. Security, liability and intellectual property are key issues as well — understand how information in large data sets is different, especially after analytics produce consolidated knowledge.

• Evaluating how applications will integrate knowledge via business intelligence (BI), process analysis (PA) and SOA. Use your understanding of potentially large data sets to develop BI and PA integration patterns that leverage a healthy service-oriented architecture (SOA) capability. Consider how knowledge from large data sets can be used to drive processes via event-driven SOA and complex event processing. Look to software-as-a-service (SaaS) and platform-as-a-service (PaaS) vendors to provide big data processing capabilities where possible when the data is appropriate to outsource.

• Updating your infrastructure technology road map and watchlist. Plan to enable horizontal scale by updating your technology road map with virtualisation infrastructure technologies and quickly put them in place when required. Evaluate data management technologies and be ready to test these in proof of concepts in support of business case development. Place big data application processing technologies on your watchlist and monitor their maturation in the context of valid business scenarios.

• Big data processing is part of an emerging “elastic application platform”.
Business demands for better value from more information will stretch, then exceed, the capabilities of traditional computing platforms. Big data processing is one such example that solves scaling issues by expanding elastically in a cloud-based infrastructure. However, the approach applies to more than just the data. Rather than classify solutions as “big data”, recognise where traditional platforms can no longer solve problems and look to distributed, elastic cloud solutions to make scalability and processing affordable.

An elastic computing platform distributes both data management and application execution processes within a cloud infrastructure, enabling affordable, flexible, high-performance computing that can scale as required. Big data processing is a core capability, but others are emerging. Firms that quickly recognise problems that can be solved by an elastic computing platform will avoid a silo approach and realise more value from their technology investment.

Brian Hopkins is principal analyst at Forrester Research