Integrated Drilling–Production Optimization Framework Using Reinforcement Learning and Digital Twin Simulation

Abstract

Abstract As drilling becomes more complex and the need for intelligent use of drilling life cycles intensifies, the petroleum industry is embracing advanced digital technologies at a faster rate. Although there has been great advancement in the automation of drilling and the use of data to optimize drilling, production, and economic performance are still considered as engineering functions separate from one another. As a result, an operating strategy that aims for optimal short-term drilling productivity can unintentionally impact reservoir productivity, shorten reservoir life, and have a negative economic effect in the long-term. To solve this challenge, an integrated optimization framework is needed that can take all drilling, reservoir, production forecasting, and economic parameters into account during the development of a field. In this study, a novel Digital Twin–Reinforcement Learning (DT–RL) framework is proposed that is capable of real-time drilling monitoring, virtual well modelling, adaptive optimization, production forecasting, and economic evaluation in one intelligent decision support environment. The Digital Twin is constantly updating the real drilling data with the virtual representation of the well, allowing real-time monitoring of the critical drilling parameters: Weight on Bit (WOB), rotary speed (RPM), drilling torque, standpipe pressure, drilling fluid circulation and bit performance. At the same time, a Reinforcement Learning (RL) agent analyzes a few of those operational scenarios on the fly and keeps on optimizing drilling strategies to achieve maximum long-term technical and economic efficiency instead of maximizing drilling efficiency on a single instant. The framework provides a feedback loop between drilling and production forecasting to allow decisions to be made to ensure the integrity of the reservoir and maximize future hydrocarbon recovery. To illustrate the feasibility of the methodology proposed, a conceptual case study was carried out with representative drilling and production data for the realistic conditions of petroleum engineering operation. The conceptual evaluation showed that the total drilling time was reduced by around 12.5 % whereas the average Rate of Penetration (ROP) was improved by 13.6 % with the integrated DT–RL compared with a conventional drilling strategy. Furthermore, the optimized drilling conditions resulted in the reduction of the average drilling torque by 11.5% and the reduction of the Mechanical Specific Energy (MSE) by 10.3%, which means that the energy used for rock-cutting is more efficient and the rock-cutting performance is improved. The Formation Damage Index estimated between 25% reduction indicating better preservation of NWI and better reservoir quality after the drilling operations. The advantages of the scheme proposed did not only apply to the drilling part of the business but also to performance in production. The conceptual evaluation resulted in an estimated 8.7% increase in cumulative hydrocarbon production and an estimated increase in recovery factor from 34.2% to 37.1% due to reduced formation damage and increased reservoir connectivity. Moreover, the highest production rate rose from 1,050 BOPD to 1,120 BOPD while the average production rate rose from 865 BOPD to 940 BOPD in the first 12 months of production. Economically, the integrated optimization strategy lowered drilling costs and improved the profitability of the project, achieving a 14.2% improvement in the Net Present Value (NPV) and a higher Return on Investment (ROI) than the conventional drilling optimization strategies. The presented results show that using Digital Twin technology in combination with Reinforcement Learning offers a viable approach to optimizing drilling operations intelligently by directly connecting drilling actions with reservoir behavior, production forecasting and economic performance. The current contribution is conceptual; it provides a technically consistent base for the realization of future implementation with field scale data, real-time sensor integration and Digital Twin operational platforms. The proposed methodology provides a potential solution for constructing next generation intelligent wells, optimizing drilling operations with autonomy, asset management for the whole well lifecycle, and thus a safer operating environment, more hydrocarbon recovery, better decision-making and economically viable development of petroleum fields.