by Duncan Irving, Teradata
In Gartner’s latest Hype Cycle, you won’t find the term “big data” listed anymore – because it’s no longer considered hype. Big data has made it all the way from its emergence in West Coast dotcoms to East Coast financial institutions, Far East manufacturing companies, and many more diverse places and industries around the globe.
Doing a quick Google search for ”big data” and Oil and Gas, you’d think that these worlds have merged now too. But no. Not only are Oil Companies not there yet, they are in danger of missing out on the whole opportunity.
Here are four serious reasons why:
1. Oil companies still manage their business data like librarians
Or should I say, museum curators?
To run the gamut from exploration to development, to production, there are many different formats of business data to be managed. Some are documents – engineering drawings from the development phase – and are managed as such. Some are physical things – rocks, fluid samples – that need to be catalogued and archived as physical things.
But a lot of it is digital data, and oil companies are not even successfully taking advantage of this data that is already available in digital format. Instead of loading digital data in an easily accessible format, oil companies store the original measurement (and any contextual data) for posterity, as a single unit.
Like a book in a library. Or a rock in a core store.
But if you don’t make the data readily available for analytics, how can you make data-driven decisions?
2. Oil companies just want to buy applications
Oil and Gas retains a strong preference of choosing to buy end-to-end data management solutions off the shelf, especially in subsurface.
Commonly we hear: “IT and data management infrastructure are not core business for us – we will not develop any custom solution”. But if you look at the industries and organisations who are benefiting the most from big data analytics and data-driven businesses –the absolute opposite is true; if what differentiates your company from your competition is how well you can turn your available data into insights, then this is core business.
It gets worse when we consider workflows that regularly need to take in data from outside the thick walls of the subsurface domain – how can you perform repetitive, integrated studies across reservoir and production data without a data management framework that spans all of Exploration and Production (E&P)?
3. Oil companies have lost their (geo)technical capability
The inventors of the Raspberry Pi were concerned our children’s understanding of computing would be how to use an iPhone or a word processor rather than how to write programmes themselves.
Tools like Petrel are replacing the holistic approach and even deterring people from testing science-driven hypotheses.
Cast your mind back to the days before the integrated workstation interpretation suites, when it was important to understand first principles. But we are losing these capabilities every day – the long-threatened “Big Crew Change” is now visible daily as oil companies contract under low oil prices.
The result is a lack of candidates to become the upstream data scientists that can discover new insights in the available data. If nobody in the Oil Company can apply the science, then analytical discovery just can’t happen.
4. Oil companies implement IT in geological time
The oil business is a strange one to outsiders. The financial numbers – both revenues and costs – are astronomical, the uncertainty is extremely high and the time to profit on a new project is long. Decisions made today may not take effect for a decade.
In the North Sea, for example, if you discover a new oil field today, you are unlikely to see first oil from it for 8 years. What will the oil price be then? The world demand? And will the technology chosen in today’s Front End Engineering Design (FEED) study still be a good choice when the field enters its second decade of production? Who knows.
In complete contrast, over in Dotcom land everything is now. Companies like eBay are constantly carrying out A-B tests on their website, constantly tweaking and changing their offering - continuous incremental improvement is the norm.
The big data technology landscape is evolving fast, and this is not the time to pick a technology and version and standardise for the future. Especially if your data formats and analytical techniques are different from the ones prioritised by the Dotcoms.
The only sure-fire way to ensure you get a big data strategy that works for you is to join in –build some systems, load some data, join the open source communities, test out new strategies, push the limits, and commit back. It certainly wouldn’t hurt your career prospects!
If – as I suspect – oil companies are not willing to show up and take part, there is a strong chance that the big data technologies that emerge the winners will not meet their needs. And that will be a huge opportunity lost.