**3.3 Machine learning model training and outcome validation**

The LR and XGB models were re-trained, using the top significant features. A drop in the model performance at the beginning of the re-training process was observed. After several iterations and hyper-parameter tuning, the predictive power of the XGB model significantly improved compared to the previous iterations; however, no improvement in the LR model performance metrics was observed. Interestingly, both models were able to identify additional new features aligned with endometriosis.

**Table 5** presents the top features identified by the XGB and LR models to be important in predicting the likelihood of endometriosis along with the statistical measures and metrics to assess the importance and significance of the features. The Chi-Square test (*p*-value) signified the importance of data elements in differentiating the target and control patients. The XGB feature importance weighed the value of features in the model in predicting the outcome. Similarly, the LR odds ratios helped to understand the odds of being diagnosed with endometriosis, given a particular medical event.

Overall, results suggest that features including 'other ovarian cyst, right side,' 'hypertrophy of uterus,' 'submucous leiomyoma of uterus,' 'excessive bleeding in the premenopausal period,' 'unspecified condition associated with female genital organs,' and 'menstrual cycle' were important in predicting the likelihood of endometriosis. The models had also flagged 'acetaminophen' and 'megestrol acetate' drugs as strong predictors of the condition.

**Table 6** shows that the XGB model performed better overall compared to the LR model. **Figure 3** shows the receiver operating characteristic (ROC) curves on the test sets for both re-trained models. The area under the ROC curve (AUC) values of the LR and XGB models were 0.87 and 0.96, respectively. Furthermore, **Figure 4** suggests that the XGB model was able to differentiate more accurately the targets from the controls than the LR model; hence, based on the final model results, the XGB model was utilized to score the qualified patients.

#### **3.4 Scoring qualified patients**

The last step of the model evaluation was to score the qualified patients to assess the model's accuracy in predicting the endometriosis onset. A sample of 9.5 million patients was identified and complete medical history was extracted for 36 months. After dataset preparation, the probability of endometriosis was estimated, leveraging the re-trained XGB model.

Probability distribution of 9.5 million scored patients is shown in **Figure 5**. Most of the predicted probability values were concentrated either toward '*0*' or '*1*'. When considering *0.5* as a threshold, the XGB model identified around 36% of the scored patients as being likely to receive an endometriosis diagnosis within the next



**Table 5.** *List of top features identified by the re-trained models.*

**119**

## *Applying Machine Learning Algorithms to Predict Endometriosis Onset DOI: http://dx.doi.org/10.5772/intechopen.101391*

#### *Endometriosis - Recent Advances, New Perspectives and Treatments*


#### **Table 6.**

*Classification metric of LR and XGB model on train and test set.*

**Figure 3.** *ROC curves of LR and XG models on test set.*

#### **Figure 4.**

*Distribution of probability on test data set for both the LR and XGB models. Figure on right side is of XGB and most of scores are grouped at extreme values.*

*Applying Machine Learning Algorithms to Predict Endometriosis Onset DOI: http://dx.doi.org/10.5772/intechopen.101391*

**Figure 5.** *Distribution of patients by predicted probability score.*

12 months. Assuming an ability to leverage the significant variables in diagnosing the condition onset, practitioners could provide focused and specialized medical care in time to their patients, thereby, reducing the risks of endometriosis and its related complications.

There is also a different way to present the data elements driving the prediction of disease onset and the scoring of patients for the likelihood of the disease. A nomogram (otherwise known as nomograph) is defined as an alignment chart or a two-dimensional diagram applied to estimate the graphical computation of a mathematical function [51]. A nomogram comprises a set of scales, where each scale denotes a selected feature of the studied population.

The nomogram tool is often employed in clinical medicine to predict patients' outcomes when considering their clinical features [52]. It is also used in clinical oncology to aid healthcare providers in their treatment decisions. It leverages regression models

#### **Figure 6.**

*Nomogram of top features to predict likelihood of endometriosis.*

such as the LR and parametric survival model as the basis for its framework [53]. For this chapter, a nomogram was selected to present a selected group of top features important to predicting the likelihood of endometriosis, as shown in **Figure 6**. The following attributes were noted on the chart as important in driving the diagnosis: 'laps total hysterect 250 gm/< w/rmvl tube/ovary,' 'other noninflammatory disorders of ovary, fallopian tube, and broad ligament,' 'other ovarian cyst, right side,' 'hypertrophy of uterus,' 'acetaminophen,' and 'pelvic and perineal pain.'

To predict the disease onset, the contribution of each feature was measured as a point score (topmost axis in the nomogram) based on the values that each feature could take with individual point scores being added to determine the likelihood of endometriosis onset. When the value of the feature was '*0*', its contribution was '*0'*points. The dotted line depicted the point score for an individual value of each respective feature with the total point being *198*, which implied a very high probability of the disease onset. Nomogram was found to be a helpful tool to graphically study the outcomes given a group of few features; however, it was also challenging to leverage it, knowing a large number of studied features [52, 53].

## **4. Discussion**

As mentioned in Section 3, the LR and XGB ML models were able to identify the top features that could help to explain endometriosis onset in advance. **Tables 4** and **5** present the important features to predict the condition onset. These features included diagnosis codes, medical and surgical procedure codes, as well as physician specialties that often support patients through their healthcare journey.

Furthermore, **Table 5** also presents the *LR odds ratio* and XGB feature importance index to aid the understanding and interpretation of the results. As noted in the above section, *odds ratios* defined the odds of being diagnosed with endometriosis when the feature changes by a unit, holding other features constant. For example, the *odds ratio* of 'uterovaginal prolapse, unspecified' was 6.40, which implied that for every additional diagnosis of 'uterovaginal prolapse, unspecified'*,* the odds of endometriosis went up by 540%. Similarly, if a patient had an additional appointment with an 'obstetrics and gynecology' specialist then the odds decreased by 47%.

As a reminder, the first part of the ML analysis was to identify the top features from an extensive list of data elements (**Table 4**). LR, XGB, and Chi-Square tests were employed to derive the final list of features to re-train the model. **Table 5** presents the most promising features with their respective significance and importance values. A number of the variables from the model were also cited in other medical and scientific journal publications, including articles from Johns Hopkins Medicine [17] and Queensland Health [18] on endometriosis signs, symptoms, and diagnosis, which confirmed the model's validity from the medical and clinical side.

In the next part of this section, the selected most important features by their respective groups were reviewed and evaluated for their relevance to the endometriosis diagnostic process. The preliminary insights for this research are available on the Research Square website. The advanced preview allowed for valuable feedback that helped to enhance the research design for this chapter.

1.Diagnoses codes: 'other ovarian cyst, right side', 'unspecified condition associated with female genital organs and menstrual cycle,' 'other specified noninflammatory disorders of the uterus,' 'excessive bleeding in the premenopausal period,' 'female pelvic peritoneal adhesions (post-infective),' 'uterovaginal prolapse, unspecified', etc. clearly showed association with the risks and symptoms of endometriosis [54]. Feature importance from XGB suggested that these features drove the model, whereas *odds ratio* from LR also indicated the direction of increase or decrease in odds of getting diagnosed with the condition. To further define the magnitude of importance, **Table 5** presents that if a patient was diagnosed with 'excessive bleeding in the premenopausal period' then the odds of receiving endometriosis diagnosis in the near future increased by 899%. Similar to these findings, Mayo Clinic articles also stated that patients might experience occasional heavy bleeding before being diagnosed with the condition [55].

2.Medical and surgical procedures: 'laps fulg/exc ovary viscera/peritoneal surface', 'laps total hysterect 250 gm/< w/rmvl tube/ovary', 'anesthesia vaginal hysterectomy incl biopsy', 'laparoscopy w/rmvl adnexal structures', 'MRI pelvis w/o & w/contrast material,' 'cystourethroscopy', etc. were also associated with the diagnosis as well treatment of endometriosis. The finding showed that for every additional procedure on 'mri of pelvis,' the odds of endometriosis increased by 1471%. Recent research from Abdominal Radiology, published by Springer Nature, also supported this claim that MRI could be more precise in the diagnosis of endometriosis compared to other diagnostic techniques [56].

As presented in **Table 5**, the procedure 'laps total hysterect 250 gm/< w/rmvl tube/ovary' had the odds ratio of 64.53, which implied that if a patient had a 'laparoscopy with hysterectomy' then the odds of endometriosis onset increased significantly. Previous studies on endometriosis also cited 'laparoscopy procedure as the gold standard' in the diagnosis process [8]. However, while the nomogram graph (**Figure 6**) also suggested that a patient was likely to get diagnosed with endometriosis post this procedure, the data element was further analyzed to understand how it might have correlated to the actual diagnoses, knowing that many laparoscopic procedures were performed to treat other female gynecological conditions. **Figure 6** shows that the feature 'laparoscope days difference' presented little importance in predicting the likelihood of the disease onset. The data element measured the significance of laparoscopic procedures in predicting the likelihood of endometriosis via calculating the days' difference between the laparoscopic procedure and the event date for both target and control cohorts.

Furthermore, the additional analysis revealed that around 60% of the target patients compared to only about 5% of the control group were diagnosed with endometriosis after a laparoscopic procedure performed on the same day of diagnosis. This finding implies that laparoscopy might not actually be a significant driver of the endometriosis diagnosis as presented in the XGB model when accounting for the time component before the diagnosis, although there were statistical significant differences between the two groups.

3.From the patient medical journey and healthcare access side, the ML models suggested that patients often consult with multiple healthcare specialists, including 'emergency medicine,' 'family medicine,' 'hematology/oncology,' 'internal medicine,' 'obstetrics and gynecology' when experiencing endometriosis-related symptoms and gynecological issues. Since, endometriosis tends to be difficult to diagnose, patients often had a number of unrelated office visits with symptoms associated later with endometriosis. This finding presented that many female patients faced substantial challenges in receiving proper care and treatment. Consequently, patients visited multiple specialists in search of answers for their signs and symptoms [57]. In agreement with these statements, both LR and XGB models presented negative weights and low importance to these healthcare providers' features, which suggested that if a patient visited these specialists more frequently, the longer it took to receive a confirmatory endometriosis diagnosis.

Furthermore, women with a history of endometriosis were found more likely to be diagnosed with either an 'ovarian cancer' or 'endometriosis-associated adenocarcinoma' in the future [21, 58–60]. With this in mind, having the ML models identify 'hematology/oncology (SPCLT\_HO),' as one of the top Board Certified specialties, further suggested that an office visit with an oncologist should be recommended for any patients presenting signs and symptoms as noted above to rule out any potential cancer risk [21, 61, 62].


Overall, the analysis results presented the important data elements to be considered when diagnosing endometriosis in women of reproductive age, to time more accurately disease onset and aid the diagnostic process. As noted in Section 3, when leveraging these features in the diagnostic process, a high accuracy prediction of the disease occurrence was identified, with the model differentiating with high precision between patients with and without the condition. Furthermore, a nomogram graphical representation could be leveraged as one of the tools to graphically predict the outcome given a set of features. Top features were utilized to showcase the practicality of the tool; however, the tool has limitations on the number of data elements that could be applied in the analysis.
