DATA MINING AND ANALYTICS

Data Mining Applications in the

Oil and Gas Industry
Ahmed Abou-Sayed, Chief Executive Officer, Advantek International

The recent proliferation of resources sis has been put on automated report- across the organization has been lack-
exploitation, in both traditional and ing, standardized presentations, and ing. The more serious gap in this dig-
unconventional basins, has led to more streamlined data transfer to cut costs ital age has been in the transforma-
upstream oil and gas industry activities for data processing and dissemination. tion of this data into information and
in more regions than ever. These newer Even when no immediate need for the knowledge that could drive manage-
activities, when added to the already data was present, the low cost of data ment actions or become the basis for
challenging work environments in such collection has driven company deci- corporate-level decisions.
frontiers areas as deep water and the sion makers to support data storing and Organizations’ imperatives toward
Arctic, place tremendous demands on sorting for later analysis. sustainability, efficiency, and opera-
the industry to work more efficiently tional challenges to meet regulatory,
and avoid risks to people, capi- competitive, and economic pres-
tal, and the environment. An sures have forced exploration
emphasis on monitoring and production corpora-
and assurance of the pro- tions to rely more heavily
duction operations during Statistics/AI Machine Learning/ on the knowledge base and
Pattern Recognition
exploitation has caused operations intelligence.
the oil and gas industry to This increases their need
enter the digital age dur- to move data toward deci-
ing these last two decades sions, improve operational
in a grand way and has efficiency and overall perfor-
Data
generated what the infor- mance, and implement tightly
mation technology (IT) indus- integrated solutions to empow-
try calls “big data.” Data acqui- er their people to make the most
sition in instrumented wells and of business opportunities. The key to
monitoring of fields and operations converting data into actionable, valu-
processes are routinely carried out Database able business insights lies today more
in both real-time and post-mortem Systems than ever in innovative IT approach-
modes. Management and use of this es—such as real-time data delivery and
big data have become critical for the collaborative work environments—
industry and its stake holders, includ- to ensure that the right information is
ing regulators and financiers. Integra- communicated to the right people at the
tion of data analytics into the prac- right time.
tice of petroleum engineers is essential Fig. 1—Data mining techniques. Currently, exploration and produc-
to establishing a vision for the oil and tion databases have the following typi-
gas industry to move toward data-based Current databases contain transac- cal characteristics:
decisions in the production and opera- tions that can be stored and retrieved ◗ Mega scale and volume—
tions arena. without relying on employees’ memo- Take for example the fact that
The largest amount of computing ries. Unfortunately, while information a large offshore field might
power in the industry so far has been banks are full, clients are not commu- deliver more than 0.75 terabytes
dedicated to the collection and storage nicating efficiently and data sharing of data weekly. A large refinery
of information (databases and ware- sometimes is inhibited by the size of will have 1 terabyte of raw
housing) as a result of the low cost and the specialized warehouses. Integra- data per day. Super majors
portability both of electronic sensors tion of the data and coincidence of the might store up to 2 terabytes
and computer memory. Great empha- information seen by all team members per day.

88 JPT • OCTOBER 2012

DATA MINING AND ANALYTICS

◗ Frequency and speed—Fast The underlying question regard- producers and water injectors in
data capture and transfer is ing the value added through data-to- the presence of high pressures
necessary because an offshore action initiatives should be clearly vis- and temperature and other
production platform might ible when one considers that large factors
deliver 4,000 to 10,000 input/ fields worldwide have seen production ◗ Defining best practices in
output points per second. declines reversed or slowed by inter- completion, depletion strategy
◗ Diversity and variety— vention and operational improvements. vs. reservoir architecture, and
Management and Significant potential exists to increase drive mechanisms
information groups in all oil rates and recovery by identifying ◗ Maximizing yields, throughput,
exploration and production key drivers for high and low oil pro- and cash flow of the operation
companies are challenged duction and implementing optimized based on different scenarios
with unfamiliar new data changes to field operating procedures ◗ Minimizing production
types that are complicated and well-stimulation methods. Data downtime and well intervention
by a lack of integration mining and analytics offer the poten- costs
between semistructured and tial to optimize solutions to many issues ◗ Ranking potential scenarios
unstructured data. and challenges to oil and gas produc- for reducing environmental
◗ Access and sharing value—It ers, including emissions and discharges
is important for operating ◗ Predicting well productivity and ◗ Selecting the best well
companies to get the right data reservoir recovery factors and stimulation method
to the right people at the right decline rate ◗ Identifying parameters for
time for efficient and timely ◗ Identifying key drivers for success, failure, performance
decisions. performance of exceptional metrics, and optimization

You drill.
 There are many examples of the past performance data provided the cor- tion, and rules induction systems. Thus,
initial success of tapping into data to nerstone for well productivity optimiza- data mining was born.
improve company operating values. tion decisions. Data mining and analytics technol-
Field intervention and workover of poor- Data mining and analytics could ogy blends traditional data analysis with
performing wells have increased pro- provide a fast and effective way to assess sophisticated algorithms for processing
duction rates by 20,000–200,000  B/D these information banks and guide large volumes of diverse types of data.
in older fields. Further operation opti- future operating decisions efficient- It has opened up exciting opportunities
mization has the ability to extend well ly. However, operators should strive for exploring and analyzing new and old
operating life and increase field total to derive intelligent recommendations types of data in novel ways. Profession-
recovery by 2–15%. To illustrate this, on the basis of known asset data and al data mining employs a suite of tech-
BP has maintained a production pla- then implement them in a cost-effec- niques to analyze relationships in a par-
teau in its Prudhoe Bay, Alaska, field tive manner based on industry stan- ticular situation. These techniques may
and extended the life of the produc- dards. Effective internal and external include statistics, pattern recognition,
er wells for an extra 7–10 years com- collaboration is also important to reach and database systems analysis (Fig. 1).
pared with operating expectations at superior economics. The selected approach depends on the
the field’s start. This work relied heav- nature of the data mining task and the
ily on thorough evaluation of historical What Are Data Mining and available data.
production data. Changes in operating Data-Driven Analytics?
procedures and water management have Digital energy experts rely on both clas- Data Mining and Analytics
helped Shell and Chevron extend aver- sical techniques, such as statistics and Tasks and Techniques
age high-water (low oil producing) well database querying, and innovative tech- Data mining and analytics tasks are gen-
life by approximately 4 years in a variety niques, such as neural networks, fuzzy erally divided into the following two
of North Sea fields. Again, evaluation of clustering, genetic algorithm manipula- categories:

You complete, stimulate and workover.

DATA MINING AND ANALYTICS

◗ Descriptive tasks—The Why Use Data Mining or production data coming in and the
objectives are to derive and Analytics? analysis results can then be uploaded
patterns (e.g., correlations, Data mining technology has the poten- into the data warehouse to continually
trends, clusters, trajectories, tial to affect all industries, including enrich that data’s value. This provides
and anomalies) that summarize design and manufacturing, resource the foundation for a continual improve-
the underlying relationships extraction and utilization, government, ment loop.
in data. Descriptive data and education. The goal of this arti-
mining tasks are often cle is to illustrate the potential benefits Road Map to Data Mining
exploratory in nature and from applying data mining and analyt- and Analytics
frequently require post- ics technology to the energy industry, Part of the operator’s effort should
processing to validate and specifically the oil and gas sector. The address ways to minimize or eliminate
explain the results. The results industry is required to react quickly to any time lag from data acquisition, real-
are presented as description or drilling, production, or other operation- time analysis, and implementation of
visualization of the system, its al issues. Analysis is enhanced and deci- resulting decisions. Reduction of dis-
components, and behavior. sions become more effective by lever- connected manual systems that deter
◗ Predictive tasks—The objectives aging data stored in an enterprise data information flow to analysts and deci-
of these tasks are to predict the warehouse that contains static and real- sion makers requires a cross-company/
value of a particular attribute on time data across existing and histori- interdivision framework for real-time
the basis of other system values. cal fields, as well as enterprise informa- operational data collection, right-time
These tasks include models tion such as financial data. These data data analysis, and seamless decision
based on neural networks, will provide context that enables the execution. High-performance data man-
discriminate analysis, and asset team to generate proactive oper- agement leverages real-time asset infor-
multiple regression analysis. ational recommendations. The drilling mation from multiple sources in con-

You produce.
junction with sophisticated analytical the user with new analysis types within standards, information transfer
tools and risk-management techniques a longer timeframe. Specialized single- standards, and production
to continually optimize oilfield perfor- purpose data warehouses—such those standards to facilitate
mance. It can help companies optimize containing seismic data, well logs, well communication across the
reservoir characterization and drilling tests, pressure/volume/temperature and business and among
and production operations by deliver- phase behavior data, and rock mechan- companies
ing a “big data” solution that improves ics—always will be used exclusively dur- ◗ Optimization of company
their ability to capture massive amounts ing specific development stages such as cash outflow between
of data, provide insights through data drilling or surveillance. capital and operational
analysis, and make better decisions. This The challenge, then, is to deliver expenditures, requiring
would help close the gap between the the following: sophisticated assessment of
vast amounts of data available and the ◗ A scalable infrastructure for operational and production
ability to turn these data into meaning- high-density data in both depth data to close planning gaps and
ful information. and time domains and a base accommodate changes in the
Such objectives require companies platform to allow high-end planning cycle
to activate data analytics capabilities analytics and provide real-time
within databanks and data warehous- advice during planning and Technology and
es to allow asset teams to leverage data execution Techniques Used
access to drive tactical and strategic ◗ Information flow from field Advances in information and communi-
decisions. Furthermore, the ability to to central functions in an cation technology since the early 1970s
assess commingled, multiwell archives integrated stream; exploration have had a great influence on all aspects
of real-time data within the same plat- and production operators of industrialization. One result is the
form (environments and tools) provides must leverage data management exponential proliferation of databanks

Make it happen.
Whatever the job, Dragon helps you get it done. Our
equipment covers applications from drilling and pumping
to storage and transportation – and all of it is severe-duty
engineered to perform under the harshest field conditions.
Plus we deliver an unmatched level of support along with our

We help. equipment, including large inventories, financing options,

expert service teams and parts and service locations in all
major hydrocarbon areas. Our mission is to help you solve
project challenges and meet your business goals. That means
we’ll come through for you, no matter what. Make it happen.

www.dragonproductsltd.com — 1-866-875-2665

U.S. owned and operated. Founded in 1963. Make it happen.

 DATA MINING AND ANALYTICS

and databases full of information too ease detection, money laundering, high- information and communication tech-
overwhelming for people to manage. performing oil wells). nology in the last 30 years have had a
As author John Naisbitt said, “We are great influence on the facilitation of
drowning in information but starving Rule Inductions. This group of meth- industrial processes using data mining
for knowledge.” ods is used to generate sets of associa- and analytics. The technique, originally
To be successful, data miners fre- tive rules that would describe the behav- developed for economic business appli-
quently and commonly employ several ior of the data set under consideration. cations, has been used in many diverse
techniques. domains. Its use in the oil and gas indus-
Predictive Tasks. Predictive tasks are try (e.g., for establishing best practices,
Descriptive Tasks. The quickest way used to predict a missing attribute of production optimization, and perfor-
to absorb the results of any analysis a new case on the basis of attributes mance prediction) is the backbone of a
is through visualization. Several tech- measured from all other cases. In addi- data-to-action initiative that includes
niques currently exist to derive pat- tion to techniques of prediction, sever- the following:
terns (e.g., correlations, trends, clus- al descriptive techniques also could be ◗ Develop a strategy and plan
ters, trajectories, and anomalies) that used for prediction, based on the con- to promote the use of data
illustrate relationships in data. To min- cept of case-based reasoning (if all other mining and analytics, artificial
imize post-processing to validate and attributes of the case are similar to a intelligence, supervised
explain the data mining results, the member of the data set, then it is rea- and unsupervised learning,
results are presented using visualiza- sonable to say that the sought attribute and other techniques in
tions of the system, its components, is similar to the nearest case). field management as a
and behavior. complementary approach to
Predictive Statistics. Similarly to traditional upstream workflows.
Visualization. This includes a variety descriptive tasks, several statistical ◗ Develop and sustain a
of graphical techniques applied in 1D, techniques are used for the prediction community of practice to share
2D, or nD space and would allow the of attributes. Multiple regression analy- ideas and promote the practice
best view of multiattribute data sets in sis, logistic regression analysis, and and application of artificial
the most suitable reduced space. Suc- discriminant analysis are examples of intelligence and petroleum
cessful examples of this task include such techniques. analytics in upstream business.
self-organizing maps and geographical ◗ Promote the establishment of a
information systems. Artificial Neural Networks. Devel- new discipline combining data
oped originally to simulate the behav- mining, predictive analytics, and
Descriptive Statistics. This includes a ior and reasoning of the human brain, oilfield technologies, including
wide variety of statistically based tech- artificial neural networks make up one curricula and training.
niques that are used to describe a set of of the powerful techniques used today ◗ Focus on promoting technology
data and that include correlations, fac- for modeling complex data sets. Conse- development and deployment
tor analysis, and survival curves. quently, the technique is able to predict to address key risks and
attributes of new cases and is widely opportunities in upstream oil
Clustering. This is a set of techniques applicable in regression analysis of non- and gas.
for studying the concept of multivariate linear systems.
data model in n dimensions and trying to Current proposed strategies to pro-
study the distance between data points Time Series Analysis. When the attri- mote data-to-action efforts in the oil
in that space in order to group them in butes of the data set under consider- and gas industry, as proposed by SPE’s
a finite or infinite set of n-dimension- ation are time dependent, this technique Production and Operations surveillance
al clusters. Hierarchical clustering and would be used to predict the future attri- track and Petroleum Data-Driven Ana-
k-means clustering are the most com- butes of the system on the basis of the lytics (PDDA) technical sections are to
mon examples of these techniques. frequency of samples. ◗ Develop and establish a
Spatial Analysis. Spatial analysis tech- PDDA advisory council in the
Anomaly Detection. This applies visu- niques are used in cases where the data management and information
alization techniques to discover unusu- set has a distribution in space, assuming discipline and a surveillance-
al or abnormal information within a that their geographical attributes could track team in the production
data set. Emphasis is put on techniques affect their behavior. Oilfield wells or and operations discipline to
directed to anomaly detection because agricultural products typically have such help data-to-action efforts
of the importance of exceptional per- attributes that could be predicted on the on direction and technology
formance in specific domains (e.g., dis- basis of relative locations. Advances in implementation

94 JPT • OCTOBER 2012

 ◗ Encourage establishment academic staff, establishing thesis top-
of a center of expertise ics for young faculty working on pro-
to harvest expertise in grams meeting the national needs of the
academia, government, country and the industry.
and industry
◗ Build a knowledge sharing Production and Operations Groups.
platform/process These groups are responsible for
◗ Develop frequent workshops improvements in well performances,
and WebEx events optimization of well stimulation, and
◗ Establish an SPE-enabled social definition of key drivers and pitfalls for
platform for collaboration high well productivity.
◗ Share best practices, methods,
and advisory council Major IT Organizations. These orga-
recommendations nizations are responsible for develop-
◗ Engage with young professionals ing educated customers with sufficient
and seek ideas and support knowledge to request and demand IT
◗ Connect with national products.
oil companies and other
international operators and Small IT Firms. These firms are respon-
champions of data-to-action sible for seeking opportunities to gen-
efforts erate business needs and service pro-
◗ Globalize and broaden visions locally and globally to address
knowledge sharing the industrial demands for consulting
◗ Interact and coordinate between and funding of development projects
various SPE and external and software. JPT
communities
◗ Interface with local SPE chapters
to promote data-to-action
efforts
◗ Interface with other societies

Technology Stakeholders
Executive and Top Management
Teams. These teams are responsible
for the assessment of fields and perfor-
mance, review of operations and identi-
fication of best practices, evaluation of
stimulation data, data classification and
clustering, identifying opportunities
to improve productivity and increase Ahmed Abou-Sayed, SPE, is chief
reserves, training professional on data executive officer of Advantek
mining technologies, generating poten- International and is the SPE Technical
tial research, and consulting links with- Director for Production and Operations.
in the professional community. Before he founded Advantek, he worked
in numerous industry positions,
Research and Technology Depart- including as a manager of geomechanics
ments. These departments are respon- at BP, technical team coordinator at
sible for development of data mining ARCO Oil and Gas, and vice president
technology, professional cadres, opera- of TerraTek. Abou-Sayed holds a BS
tion and engineering staff education, degree in mechanical engineering and
and facilitating collaboration within the design from Cairo University and an MS
IT industry and the operations staff, degree in solid mechanics and a PhD
initiating research and development degree in engineering mechanics from
efforts for the engineering and the IT Brown University.

JPT • OCTOBER 2012