According to the IDC, by 2020, there will be enough data in the world to fill a stack of tablets equivalent to 6.6 times the distance between the Earth and the moon. While the future looks exciting through the lens of big data, organisations have a lot to consider before they dive in. 

The futurist on big data

In big data, futurists see greater efficiencies, increased productivity, reduced costs, and business growth. For the futurist, any business that does not build its strategy around data is dead in the water. Nearly 80% of enterprise executives in one Accenture study said companies that do not embrace big data face extinction.

Artificial intelligence and machine learning promise to transform businesses, reconfigure workforces, optimise infrastructure behaviour, and blend industries through radically improved decision-making and process automation. IDC expects global spend on AI to reach $57.6 billion by 2021, and 72% of business leaders told PwC that AI will be a ‘business advantage’ in the future. 

With big data and analytics, businesses can drive digital transformation, redefine the customer experience, and create a new sense of relevance, rather than just providing a product or service at the lowest price. But, for the realist, it’s more complicated than it sounds. 

The realist on big data 

In big data, realists see information overload – and it’s growing rapidly. By 2020, 1.7 megabytes of new information will be generated every second, for every human being. By then, there will be 44 zettabytes of data in the world, up from 4.4 zettabytes in 2015. 

But data in and of itself is useless. Organisations need to know what data they have, where it resides, and what questions to ask. Surprisingly, few organisations get it right: While 95% of Fortune 1000 businesses have undertaken a big data project in the last five years, only 48% have achieved measurable results. And only 15% of businesses have deployed their big data projects to production. 

Data and project sprawl

In most organisations, data resides in a number of systems. These silos make it difficult to know what data a business has, what state it’s in, and how to scrub and prepare it for analysis. 

It’s also common to find multiple departments within an organisation running their own business intelligence (BI) projects. But this disjointed approach does not produce the kind of insights that transform businesses. 

Only when systems are integrated, and all data can be accessed in one place, will real opportunities emerge. But legacy technology and employee resistance is getting in the way. 

Technology, tools, and culture 

Many businesses run on legacy systems that are not powerful enough to process data on the scale needed today. Yet they’ve invested a lot of money in their infrastructure, and ripping and replacing it will be an expensive, disruptive, and unrealistic exercise. 

Then there’s the BI tools, many of which can only be understood by data scientists, who are in short supply and high demand. While there are proprietary and open source tools that are geared towards the business user, businesses still struggle to get buy-in from everyone in the organisation as they shift towards data-driven cultures. 

Security 

As the new currency, data is extremely valuable to cybercriminals and rogue employees who will find any opportunity to steal sensitive information for their own financial gain. 

Further, authorities are clamping down on businesses that don’t adequately collect, process, store, and discard personally identifiable information. The Protection of Personal Information Act and the EU’s General Data Protection Regulation propose massive fines for businesses that are careless with their data. 

Despite these challenges, futurealists are still optimistic. 

The futurealist on big data 

For the futurealist, any BI project is ultimately about empowering teams to better serve customers. For this reason, they focus as much on people development as on technology enablement. 

They believe there’s no point gathering data if the business has no clear use for it. Because of the sheer volume of information available, organisations need to know what data to collect and what to disregard. It’s not possible – or necessary – to collect everything. 

People 

Big data success depends on a solid data governance framework, ideally overseen by a chief data officer (CDO), who is responsible for how data is prepared, stored, secured, and accessed. Gartner predicts that, by 2021, the CDO role will be as mission-critical as the IT, business operations, finance, and HR functions in 75% of large enterprises. 

But the CDO function is just one of many roles that make big data projects possible. Futurealists know that the skills shortage is a problem and that the answer, for most organisations, is in their own people and citizen data scientists. 

As more business functions are automated, organisations will need well-rounded individuals with a blend of business and technology skills; people who know what to look for in the data and who can trust their intuition to either follow a hunch or leave room for the unexpected to emerge.

By Matthew Kibby,

Vice-President, Enterprise, Africa & Middle East at Sage