The term “quantitative assessment” in the title refers to the consistency check of the finally selected static model with actual data and geological understanding, and to the relative assessment of the model accuracy under current conditions of available data and cognitive capacity, including but not limited to quality control for a certain process of modeling.

New techniques of 3-D geological modeling have developed by leaps and bounds in recent years. By contrast, papers on discussing how to systematically evaluate the quality of 3-D geological models are considered rare, especially the quantitative evaluation of the finally screened geological model. Professionals seem to pay more attention to modeling methodology than model reliability. Overseas scholars are accustomed to quality control of 3-D models from the perspective of accuracy assurance or reducing uncertainties^[1,2,3]. However, its primary purpose is not the quality evaluation of the final model. Chinese colleagues prefer discussing some major issues which may have a significant impact on the quality of final model, verifying the model with static and dynamic data^{[4,5,6,7,8,9]}, but it is still a qualitative evaluation as a whole. Although there are highly visible efforts in the field of quantitative evaluation^[10], the single-parameter evaluation is not robust. There are two reasons for the lagging development of quantitative evaluation. Firstly, due to the complexity of subsurface geological settings in oil or gas fields and different data conditions, there is no comparability, and thus the universal framework for evaluating model quality is currently not available in the public domain^[5]. Secondly, on account of different types of reservoirs and different development stages, it is difficult to select common evaluation parameters and establish uniform quantitative standards. However, fundamentally, it is the traditional way of thinking without differentiating reservoir types that restricts the establishment and development of the quantitative evaluation criteria of 3-D geological model. If we narrow the evaluation scope down to the same reservoir type, or even to the same research target, the comparability and practicability of the quantitative assessment will be emerging, such as the following two cases: the quantitative assessment can enable different geological models to be compared, which have been created by different researchers using the same data, so as to provide options for decision makers; it also facilitates the exchanges between inspectors and geo-modelers to grasp jointly key points of assessment and reach consensus. Compared to qualitative evaluation, the multi-parameter quantitative assessment not only has objective and comprehensive advantages, but also can identify the aspects to be improved according to the deducting points. In short, it is of great importance to establish and develop the quantitative assessment standard and to evaluate models in accordance with it. It is a necessary supplement to the workflow itself. In the future, the test of the reliability of geological models will become a research hotspot in the field of reservoir modeling and develop towards refinement and standardization^[8].

Oil and gas reservoir description can be categorized into four types from the perspective of geological modeling: oil-reservoir description of large stable sedimentary system, oil-reservoir description of complex fault block, low permeability oil/gas-reservoir description and fractured oil-reservoir description^[11]. Taking the oil reservoir in complex fault block as an example, we discuss the multi-parameter quantitative evaluation method of 3-D geological models.

We started the research with a wide investigation of characteristic parameters which are most likely to impact the quality of oil-reservoir model for complex fault block, and then designed a survey form, involving evaluation system, methods, associated parameters and weights, to collect the feedback from ten experts with over 30 years of work experience and high popularity in the petroleum industry, including four geologists, two geophysicists, two petrophysicists and two reservoir engineers. The multi-parameter quantitative evaluation criteria were established based on the investigation and the consultation. The evaluation strategy includes four points as follows:

(1) The quantitative evaluation system is composed of three main parts, namely, data verification, geological understanding and process check. Data verification is to verify the accuracy of local prediction based on actual data in the field (hard data). Geological understanding is to examine whether the global estimation honors geologic rules and prior understanding (soft data). Both of them check the model quality from a view of direct information and indirect information respectively, and complement each other, which are the focus of quantitative assessment. Therefore, they are weighted by 60 points and 20 points respectively in centesimal system. To avoid occasionality, process check is indispensable, weighting 20 points.

(2) The evaluation adopts PCA (Principal Component Analysis) method^[12], which has the ability to incorporate all experts' opinions on characteristic parameters and their corresponding weights. The characteristic parameter here refers to the representative parameter which can describe an oil or gas reservoir. Apart from representativeness, attention should be paid to avoiding repetition or correlation across the selected parameters.

(3) As the model evaluation standard is established for complex fault-block oil reservoir, the total weights of the characteristic parameters related to structural model should not be less than one quarter of the total score, that is, 25 points.

(4) Following determination of the characteristic parameters and their weights, we accordingly set the evaluation threshold values of each characteristic parameter, and subsequently obtained the final evaluation result based on the cumulative score, which is typically categorized into three levels, that is, low, medium and high level.

2.1.1. Verification methods

The most direct verification is by means of hard data, which are usually divided into four categories: (1) New well test. (2) Blind test. Blind tests can be representative and convincing, provided that blind wells are distributed evenly and comprise cored well(s). (3) Well performance test. The injection-production response is the special case. (4) Numerical simulation test. The quality of static model is evaluated according to the initial fitting error in history matching^[4-5,10]. Generally, the first two belong to static methods, which are comparatively rapid and straightforward, but lack of representativeness due to local tests, whereas the latter two are dynamic methods. Although the representativeness of the third method is somewhat improved, it just keeps reservoir connectivity in view. The advantage of the forth method is that it involves a great number of wells and is therefore deemed to be more representative; however, it is time consuming. As a result, it is wise to use the four methods comprehensively.

Moreover, streamline simulation falls into the forth category, and has been widely utilized in recent years^[4,5], because it does not necessarily coarsen the model, instead, with given fluid parameters, it can quickly model the flow path of the injected fluid based on the reservoir heterogeneities of the existing model, and perform a rapid assessment in combination with the injection-production response in the oilfield. As for “the consistency check of probability distribution”, an approach to examine if the final model honors the frequency statistics from the well-log data, strictly speaking, it is not the quality evaluation of the final model despite using hard data. On the country, it is just an important step during quality control, and an essential procedure for developing a high-quality model^[13].

2.1.2. Characteristic parameters and their weights

Data verification is subdivided into the static and the dynamic. Six characteristic parameters were selected in this study, including five static parameters (complex fault contact relationship, target zone depth, lithofacies and depositional facies, net pay and permeability), and one dynamic parameter (initial fitting error of water cut). This may be attributed to the following:

Two characteristic parameters regarding structural model are the complex fault contact relationship and the depth prediction of target zone^[14,15]. Faults not only determine reservoir compartmentalization, but impact fluid flow in the reservoir. At the development stage of oilfields, low-grade faults have been increasingly drawing attention^[16,17].

The characteristic parameter selected in facies modeling is the lithofacies or depositional facies, which can be described as lithologic units that include a series of geological characteristics. Facies model can be considered as an elementary component of reservoir model^[18], because facies are highly linked with the petrophysical properties that directly determine the hydrocarbon pore volume and fluid flows^[19].

Two characteristic parameters selected in property modeling are net pay and permeability. Net pay is a critical parameter of reserves estimation, typically in the form of NTG ratio in the model. In addition, the goal of drilling campaign is to find oil-bearing layers as thick as possible. Hence, it is of practical significance and commercial value to take net pay as a characteristic parameter. Permeability is the most direct parameter dictating the ease of fluid flow and ultimate recovery. Nevertheless, its range of variability is the largest^[20], so it is the most difficult to predict.

The characteristic parameter in the numerical model is the initial fitting error of water cut. The data to be fitted in history matching frequently include pressure, fluid production rate, oil production rate, and water cut of the oilfield or single well, and the emphasis is to fit initial water cut and primary variables during the late period of production history^[21].

The above six characteristic parameters are all important for quality evaluation of geological model based on real data. Consequently, according to the opinions of 10 experts, the weights of them were equally given 10 points.

Comprehensive geologic studies play the core role in reservoir modeling^[22]. Therefore, the final static model should reflect the prior understanding and replicate the geologic reality^[23,24]. Two characteristic parameters, namely the variogram parameter selection and the final static model selection, were chosen according to the following reasons:

Different geostatistical techniques have seen an extensive application in 3-D geologic modeling in recent years^[25,26,27]. Geostatistical simulation algorithms are mostly grouped into two families: variogram-based techniques and variogram-free techniques. It is reported that almost 90% of the reservoir-characterization studies use variogram-based geostatistical techniques to create a 3D model^[26]. Proper selection of variogram parameters (range, nugget, azimuth, etc.) is a key to obtain a geologically sound reservoir-characterization model^[5,28]. Unfortunately, variogram modeling and interpretation are often performed hastily or even skipped altogether. Considering that it is interrelated to the already selected characteristic parameter, i.e., lithofacies or depositional facies, its weight was assigned 5 points.

For limited data cases, analog data can be used to establish variogram models^[3,5]. For variogram-free techniques, other parameters can be selected instead. For example, training image can serve as a characteristic parameter in multiple-point simulation.

The final static model selection is a characteristic parameter which can fully express the geological understanding accomplished in the early phase of study. The original oil in place (OOIP) is conceived as one of the most important geological insights. Under the same condition of hard data and geological understanding, if the modeling methods are appropriately employed, the P50 reserves (the corresponding reserves at 50 percent of the cumulative probability, also known as probable reserves) derived from stochastic simulation should be equal to or close to the prior reserves in the deterministic method; the prediction error should be commonly no more than 10%^[6]. If the inference bias is too large, it is likely that there is a problem of selecting calculation parameters or modeling methods^[4], because both methods use the volumetric method and evaluate reserves on the basis of average values: all the calculation parameters in the deterministic method are taken as the arithmetic mean, and P50 itself is median probability usually used as the base case of geological reserves in the probabilistic method^[29]. The relative error of reserves estimation has a veto on the evaluation of model quality, so the maximum weight, 15 points, was determined among all characteristic parameters.

For exploration and production, probabilistic methodology has more advantages over deterministic methods as to reserves estimation, partially because the former adequately considers the uncertainties of calculation parameters through statistical tools, which leads to a range of reserves rather than a fixed value, and avoids false precision^[30]. The reserves estimation obtained by probability calculation is more comprehensive and able to better assess reserve risk^[29,31]. However, it does not mean that P50 reserves are more dependable than the prior reserves. This is a comparison of two methods, i.e., probabilistic method and deterministic method, instead of a contrast between two numbers. Furthermore, the selection of final static model should take integration of dynamic examination into account^[27].

To assure reliability, the systematic quality evaluation should not only emphasize the result validation, but also value the process check, that is, the quality control of important steps during the modeling process. We adopted spot check to make the evaluation system simple and practical. The selected five characteristic parameters in this study include grid design, facies modeling method, seismic constraint, saturation calculation method and reserve impact factor analysis. The selection reasons are listed as follows:

Grid is the minimum description unit of model and the foundation of modeling. The grid boundary, size, direction and trend are defined according to study purpose and project area. It is significant to check the number of irregular grids near faults, examine whether there exist intersections between upper and lower grids, and whether the grid cell is negative. Weight it by 5 points.

Facies modeling is a crucial part of reservoir modeling. It usually requires researchers to analyze the adaptability of modeling techniques and to attempt to use some innovative or integrated approaches. In view of importance and time cost, this characteristic parameter was weighted with 7 points.

With limited well control, it is important to use seismic data to constrain models, which contributes to improving the model quality to a large extent^[24,32-35]. The integration of seismic inversion results in property modeling is now common practice^[36]. The weight of seismic constraint is 4 points.

It is necessary to examine the modeling methods of initial water saturation, because the spatial distribution of this petrophysical property is controlled by gravity in a reservoir, in which the initial water saturation gradually decreases from the free water level up to a given height of oil column^[37,38]. The saturation calculated with the capillary pressure is very close to the one from sealing core analysis^[39]. Therefore, the capillary pressure method, the J-function method or the saturation height function method should be preferred in massive reservoirs with distinct transition zone. The saturation calculation method weights 2 points.

Uncertainty in reservoir characterization and modeling is caused by the uncertainty of input data and the uncertainty in the inference. Uncertainty analysis, therefore, should include studies of data inaccuracies and inference uncertainty from data to estimation of reservoir properties^[40]. Inference uncertainties usually include conceptual, interpretational, and methodological uncertainties. It is more straightforward to assess uncertainties of some “composite” reservoir variables, such as hydrocarbon volumetrics, using stochastic simulations^[40]. Uncertainty analysis is an integral part of high-quality modeling and also a research trend, which is in general moving away from an “honor the data” paradigm to an “honor the data and respect/incorporate uncertainty” paradigm^[41]. Chinese scholars should pay enough attention to it. The weight of reserves impact factor analysis is 2 points.

In summary, 13 characteristic parameters were selected altogether from data verification, geological understanding and process check, among which the cumulative weights of parameters related to structural model amount to 25 points (including 20 points from data verification and 5 points from process check), meeting the requirement of the third item in the evaluation strategy.

The three-level threshold values of each characteristic parameter need to be optimized when the standard is initially formulated for a certain type of reservoir in a given region at a particular stage, so as to avoid losing the significance of relative evaluation if the standard is too rigorous or too loose. Hence, it is desired to firstly estimate the prediction difficulty of a given characteristic parameter according to data quality and subsurface heterogeneities, and then refer to industry standard, the precision level of analogies and work experience, and finally use in-situ data to determine suitable threshold values through trial and error. In the following case, for the reservoir at a depth of -1 400 m with a well spacing of 300 m, the accuracy standards of well-top prediction vary from the data condition with to without 3-D seismic survey. If available, the average absolute error of less than 1 m can be set as high level; if not, less than 2 m works. Similarly, in an attempt to amplify the differences of the three-level evaluation results, the evaluation standard of each characteristic parameter is established as full score, half score, and one point or zero point for high, medium, and low level respectively. The final evaluation result is grouped by the cumulative score into three levels: high level when the score is greater than or equal to 80 points, medium level if the score lies between 60 and 79 points, and low level as the score is less than 60 points.

The evaluation standard of characteristic parameter is not invariable in different development phases. For example, in the conventional water-flooding sandstone reservoir of the western Zone 7 in Gudong Oilfield, the reservoir engineers statistically analyzed the water-cut fitting errors of 312 wells at the low-medium water-cut stage and that of 420 wells at the high-extra high water-cut stage separately, and studied the adaptability of evaluation criteria^[10]. At the low-medium water-cut stage, due to the rapid rise of water cut, the variability of water cut is large. So is the fitting error. For instance, there were only 54.5% of wells with a relative fitting error of less than 5%, whereas there were 86.5% of wells with that of less than 10%. Obviously, it is too strict if the error of less than 5% is taken as the standard of high level, and it is appropriate to take relative error of less than 10%. By contrast, at the high-extra high water-cut stage, the variability of water cut is relatively small, and so is the fitting error. The number of wells with a relative fitting error of less than 5% accounts for 89.5%, while the number of wells with that of less than 10% does 97.4%. Apparently, it is too loose if the error less than 10% is regarded as the standard of high level, and it is suitable to take relative error of less than 5%. Similarly, as far as the cumulative score is concerned, the threshold values of three- level evaluation also vary from different development stage.

On the basis of the 13 characteristic parameters and their weights, we generated the quantitative evaluation standard of 3-D geological models for complex fault block reservoirs and presented a case (Table 1).

Block F, located in the south of Songfangtun Oilfield in Daqing, China, is a typical complex fault-block sandstone reservoir (Fig. 1). It is characterized to be a monocline plunging towards north east as a whole and a series of low relief structures are generated inside. The producing zone is the lower Cretaceous Putaohua oil layers in Segment 1 of Member Yaojia at a depth of 1 330-1 430 m. The reservoirs principally consist of delta front sandbodies and Putaohua oil layers is vertically divided into 13 units. The rock properties are marked by medium porosity and medium to low permeability. Porosity ranges from 12% to 26% and on average 18%. Permeability is (3.47-401.52)×10^-3 μm², on average 10.29×10^-3 μm². Block F was put into production in February, 2007, adopting a 300 m×300 m inverted seven-spot water flooding pattern. By the end of 2015, 56 producing wells and 29 water injection wells had been drilled. The recovery factor of Block F was 14.6%, the annual production rate was 1.08%, and the total water cut was 67.5%.

A wide variety of data were available in this case, including 85 well logging data and test data, 3 core well data, 3-D post-stack seismic data, and top structural map of Putaohua oil layers as well as production data.

According to the evaluation criteria, the 13 characteristic parameters were measured item by item, and the final model scored 92 points out of a possible 100 (Table 1). The deduction items and the full score items were analyzed as follows:

4.2.1. Deduction item analysis

In terms of single item score (Table 1), there are three items deducted, including the initial water-cut fitting error (5 points), the saturation calculation method (1 point) and the reserve impact factor analysis (2 points).

The statistical results show that, among 54 producing wells, the number of wells with the water-cut fitting error of less than 5% accounts for 64.8%,. According to the evaluation standard, it belongs to medium level, so 5 points are deducted.

Without applying proper approaches to deal with the oil- water transition zone caused by low permeability, the saturation models were generated just by carrying out the Sequential Gaussian Simulation (SGS) according to the initial water saturation derived from porosity above oil-water contact, which was of medium level, so 1 point was deducted.

After reserves estimation and model selection, sensitivity analysis or impact factor analysis on reserves was not performed unfortunately, and 2 points were thereby deducted.

The first deduction is normal in consideration of its prediction difficulty. However, the latter two should have been avoided. The problems occurred because the geo-modelers had not paid enough attention to the technical development and research trends in the field of geologic modeling.

4.2.2. Full score item analysis

One case was selected from each part of assessment system (data verification, geological understanding and process check) and analyzed, that is, the complex fault contact relationship, the final model selection and the facies modeling method selection respectively.

4.2.2.1. Complex fault contact relationship

Checking the contacts between faults, especially complex faults, is one of the major issues to evaluate the model quality of complex fault-block reservoirs. The X-shaped faults was checked firstly in this case, involving three faults such as F10, F35 and F22 (Fig. 1). The model reveals that the generated faults clearly exhibit the shapes of fault surfaces and the fault contacts (Fig. 2a), and the fault surfaces also maintain consistency with the fault sticks of seismic interpretation (Fig. 2b). Secondly, the contact relationships were investigated between faults and stratum. There are two faults, namely F1 and F62 (Fig. 1), which do not intersect at the top of Putaohua oil layers (Fig. 3a), but cross at the bottom (Fig. 3b). This detail was reproduced in the model. Based on above analysis, a conclusion is drawn that the fault contacts are reliable. Consequently, this item was given full score.

4.2.2.2. Final model selection

The final model selection is a characteristic parameter that can comprehensively incorporate the geological understanding, as discussed previously. Only when selecting a geologic model with relatively small error of the prior reserves, can one profoundly perceive the essence of geological understanding. Besides, the selected model is required to honor the oilfield’s production performance^[27]. A total of 50 realizations of equal probability were generated with the stochastic simulation technique. P50 reserve was obtained by running Monte Carlo simulation, to recognize the 23rd case which corresponds to P50 value. Other four realizations (corresponding to the 38th, 32nd, 16th and 42nd; the former two are slightly smaller than P50 and the latter two are marginally larger than P50) were also selected, and then the final model was screened out by streamline simulation. As a rule, the streamline density is strongly correlated with sweep coefficient of water flooding^[42], regarded as an important factor to judge well-control degree and interwell reservoir connectivity. The geo-modelers compared the five corresponding streamline simulations.

On the whole, it is considered that the flow characteristics of the 38th realization can explicitly incorporate the injection-production status in the north, center and south. In other words, the streamlines of the 38th realization nearly cover the northern production area, implying interwell reservoirs are well connected, and the well pattern is relatively reasonable; the sparse streamlines in the central production area indicate the reservoir connectivity gets worse and the efficiency of water flooding is unsatisfactory; the situation of the southern production area lies in between (Fig. 4).

In terms of local region, the trend of the production curve of Well FI-158 is in overall agreement with that of the injection curve of Well FH-154, showing that the reservoir connectivity is pretty good between two wells (Fig. 5). The streamline model of the 38th case better captured the injection-production response because its flow path properly matches the shape of the main river channel (Fig. 6).

In a word, the 38th realization is the optimal static model. The estimated reserves are 242.7×10⁴ t; the relative error is -1.5% compared with the prior calculation (246.4×10⁴ t). As a consequence, this item acquired full score, 15 point.

4.2.2.3. Facies modeling method selection

Efforts were made to investigate four facies modeling approaches undertaken to characterize the main river channels in this case, including sequential indicator simulation (SIS), object-based simulation (OBS), multiple-point simulation (MPS), coupling seismic constraint and facies boundary simulation (CSFS). The result suggests that the river channel’s continuity simulated by the SIS method can meet requirement (Fig. 7a). The OBS method is very flexible in terms of replicating complicated channel’s geometries, but fails to honor well data completely (Fig. 7b); the MPS method allows for replication of complex geometries of main channels more than channel’s continuity, which leads to a remarkable difference between the output and the underlying reservoir geology (Fig. 7c). The CSFS method integrates seismic constraint and facies condition into the sequential indicator simulation, and a blind test demonstrated it could honor common geologic principles horizontally and facies recognition from well logs vertically with a smaller error compared with SIS (Fig. 7d). Additionally, the computation speed is listed from fast to slow as follows: SIS, CSFS, OBS and MPS. As such, the CSFS method is believed to be the optimum technique to build the lithofacies model and the depositional facies model for study area (Fig. 8).

It is the traditional way of thinking without differentiating reservoir types that limits the establishment and development of the quantitative evaluation criteria of 3-D geological model. If we reduce the scope of evaluation to the same reservoir type, or even to the same research target, the feasibility of the quantitative assessment will be present.

The quantitative evaluation system comprises three main parts, namely, data verification, geological understanding and process check. The precise result is undoubtedly to be expected, but the sound process should also be emphasized. Apart from representativeness, care should be taken to avoid repetition or correlation across characteristic parameters; the three-level threshold values of each characteristic parameter need to be optimized, lest the standard lose the significance of relative evaluation if it is too strict or too loose.

As the quantitative assessments of 3-D geological model are still experiencing with ongoing research, some key factors affecting the quality of models are not involved in the system, such as the reliability of input data mentioned previously. In practice, quantifying the evaluation of input data can be a real challenge because it belongs to another dimension in terms of current characteristic parameters. How to address this issue needs further studies.