Prediction of success of slings in female stress incontinence, statistical and AI modeling

Stress urinary incontinence (SUI) is the most prevalent type of urinary incontinence (UI). Age, parity, and obesity were globally considered risk factors for the development of SUI. Other reports demonstrated factors such as hysterectomy, medical co morbidities, smoking, DM, and major depression^1,2,3,4.

While pubovaginal sling (PVS) is the gold standard, the use of mid-urethral slings (MUS) became the most popular procedure since Ulmsten published his study on tension-free vaginal tape (TVT)^5,6.

In addition, some studies demonstrated that primary and secondary outcomes of PVS and TVT were better than those of TOT^7,8.

Different predictive models for prognostic and diagnostic purposes were created using “experience”. Logistic regression (LR) and artificial neural networks (ANN) are two examples. These models are based on 2 distinct yet similar fields; statistics and computer science⁹.

Compared to LR, ANN has various benefits: requiring less statistical training, detecting potential interactions between predictors, and complex nonlinear correlations. However, drawbacks include its black-box nature, the need for more complex computing procedures, the empirical nature of model construction, and the possibility of over-fitting the data, which could compromise the model’s ability to generalize¹⁰.

Support Vector Machines (SVM) is an algorithmic application of concepts from statistical learning, which helps constructing reliable estimators from data¹¹. By resolving a constrained quadratic optimization problem, SVM construct the best separation boundaries between data sets. Different levels of nonlinearity and flexibility can be incorporated into the model by employing different kernel functions. SVM is developed from sophisticated statistical concepts and limitations on the generalization error can be eliminated^12,13. Our study aims at exploring different models which can help in predicting the outcome of MUS. We believe that the application of AI in this end is both timely and clinically relevant, as many studies^14,15,16 failed to predict the success of MUS based on usual statistical methods.

All women underwent MUS in our facility from January 2002 to January 2020 with a minimum follow- up of 1 year, were retrospectively studied. All methods were performed in accordance with the relevant guidelines and regulations.

Inclusion and exclusion criteria were similar to previous report⁷. 257 patients were contacted by phone and asked to attend an outpatient clinic visit. Informed consent was obtained from all patients. The study design and protocol were approved by the local ethical/scientific committee of UNC. As the data were retrospective and no disclosure of patients’ identities were made, UNC IRB does not mandate informed consent forms to be collected.

Confounding factors were: age, body mass index (BMI), parity, previous pelvic surgery, pre-operative urodynamics (UDS). Follow up visits included per vaginal examination, stress test, pad test, post-void residual (PVR) and symptom scores.

The primary outcome is the construction of a prediction model that selects the patient with the best success rate. Cure is defined according to objective criteria (a negative stress test, a negative 1-h pad test and no retreatment) and subjective criteria (self-reported absence of symptoms, no leakage episodes). Failure was defined as persistent stress component.

Patients’ data were retrieved and reviewed regarding demographic data, surgical history, preoperative examination, pre-operative UDS, type of the sling, concomitant repair of prolapse, and BMI which was classified into values of more or less than 30. Parity was classified into values of more or less than 3. Abdominal leak point pressure (ALPP) was classified into 3 grades; > 90 (grade 1), 90—60 (grade 2) and < 60 (grade 3)¹⁷. Pre-operative bladder capacity was classified as normal or low (< 250 ml). The type of sling used was TVT, TOT or PVS.

Statistical analysis was carried out using SPSS version 24. In phase I, variables were categorical and Chi-Square test and McNemar’s test were used. Spearman’s correlation coefficient was used to define the association between variables. Binomial logistic regression analysis was used for the evaluation of the impact of independent variables on the prediction of the outcome of surgery.

Phase II included analysis of data through AI model (ANN and SVM). The output was divided into success or failure.

ANN consists of neurons in several layers, where each neuron is considered a mathematical unit. Each input is given a weight. During training, the weights and biases of all neurons get modified as the training continues with the target of reaching the best accuracy. Our model consists of 3 layers: input layer, a hidden layer, and an output layer. The first has 10 neurons (the number of input features). The hidden layer has 13 neurons and the output layer has 2 which correspond to the output classes 0 and 1.

The first 2 layers use the Rectified Linear Unit (ReLU) as the activation function while the output layer uses the Log-Softmax.

In our study, we used the negative log-likelihood loss:

With this loss, we can calculate the gradients for each weight and bias of each neuron with a proper optimization function. We used the adaptive moment estimation (ADAM) optimizer, which modifies the weights and biases with the gradients calculated from the loss function. We trained the model for 50 epochs and used that number for quick redoing of iterations.

Phase I

Analysis of data showed no statistical significance regarding different age groups or BMI groups. However, cases with BMI over 30 had the highest failure. Parity was statistically insignificant even though more failures were noted when parity was > 3. Women with previous pelvic surgery showed higher failure rate than those who had no pelvic surgery (P value < 0.05) (Table 1).

Analysis of pre-operative UDS showed statistical significance regarding ALPP and pre-operative maximum bladder capacity. ALPP is thought to be a crucial predictor for the outcome of MUS. Patients with ALPP below 60 cmH2O showed higher failure rate than those with over 60 H2O (P < 0.05). Also, low bladder capacity before surgery was accompanied with higher failure rate than those with normal capacity (P < 0.001). A moderate correlation was reported between pre-operative bladder capacity and the outcome of surgery (r = 0.4).

Increased failure rate was noted in cases having TOT. (P < 0.05), while concomitant repair of POP, had no effect on the outcome of surgery. (Table 1).

Logistic regression

Binomial logistic regression was performed including 151 patients in total, to assess the effects of age, BMI, parity, previous pelvic surgery (PPS), abdominal leak point pressure (ALPP), pre-operative maximum bladder capacity (Pre-Cap), uninhibited contraction (UC), concomitant repair of POP (CRPOP), and the type of the sling (independent variables) on the likelihood of success. The LR model was statistically significant in prediction of the results with an accuracy of 90.7% [X² (11) = 46.24, P < 0.001]. According to the accuracy of this model, the area under the ROC curve was 0.93 (95% CI 0.883 to 0.977) (Fig. 1), which is important in discrimination according to Hosmer et al.¹⁸ with overall accuracy 90.7%, positive predictive value of 61.5% and negative predictive value of 93.4%.

Many predictive variables of this model were found statistically significant in the prediction of dependent variable as shown in Table 2, which demonstrates that: patients with previous pelvic surgery were 8 times more likely to have failure than those without, patients with < 60 ALPP had 24- and 6-times higher odds to have failure than patients with > 90 and 90–60 ALPP, respectively, patients with low pre-operative maximum bladder capacity had 29 times higher odds to fail surgery than those with normal capacity. Spearman’s correlation coefficient between bladder capacity and outcome of surgery was strong with r = 0.415 (P < 0.001) (Fig. 2) Patients with TOT sling type had 22 times higher odds to fail surgery than those with PVS.

Phase II

Despite the promising results of binominal LR model, we explored ANN in an attempt to reach more accurate predictability.

The training set consisted of the records of 90 patients. After the network was trained accordingly, it predicted the outcome in a further 61 patients who comprised the testing set. The network was blinded to the output in the testing set. We evaluated the sensitivity and specificity of training as well as the testing sets using a confusion matrix. Figure 3 depicts the construction of the ANN.

Because of data imbalance and the relatively-small sample size, the neural network plateaued at 90.14% accuracy and F1-Score was 94.8%. Accuracy of ANN in discrimination between success and failure rendered an area under ROC curve of 0.7232. (Fig. 4).

The ANN model was not able to predict a single failure case. Therefore, we run our data via SVM and calculated accuracy, sensitivity, specificity, positive and negative predictive values. Figure 5 demonstrates graphic representation of SVM model.

The accuracy of the SVM model in the training set (using the records of 151 patients) was 93% accuracy and 96% F1-score. Table 3 shows the confusion matrix of SVM. In this model, accuracy is acceptable but its capability of prediction of failure is still unsatisfactory as positive predictive value did not exceed 40%.

In our study, cases were done only by 2 surgeons, accordingly the effect of the operator as a confounding factor could have been eliminated.

BMI ranged from 21.6 to 34.7 and despite of having statistically insignificant impact on the outcome. Our results were supported by the study of Bach et al.¹⁹ who reported low incidence of failure among patients with similar BMI range. Patients with BMI ranged from 35 to 50 reported higher incidence of failure and they should be offered weight loss first before surgery. This is going along with what Lee et al.²⁰ have found in 138 women with SUI who underwent TVT and concluded that high BMI, low ALPP, and high grade of incontinence may impair the cure rate of the TVT.

Salhi et al. did a systematic review to evaluate the effect of previous pelvic surgery on the outcome of MUS. They conlcuded that one of the main predictive variables for adverse events following MUS was previous pelvic surgery [OR: 3.7 (CI 95%: 1.14–12.33); P = 0.029]²¹ and this is similar to what we found in our study where previous pelvic surgery is associated with higher failure rate [OR: 7.847 (CI 95%: 1.642–37.504); P < 0.05].

Nager et al. analyzed 260 cases of failure post MUS and reported that when ALPP is less than 86 cm H2O, risk of failure is 2-folds, regardless of the sling type [OR 2.23 (CI 95%: 1.20 – 4.14)]²² which is similar to what we reported in our study, when ALPP was less than 60 H2O, the chance of failure was 24- and 6-times higher odds to have failure than patients with > 90 and 90–60 ALPP, respectively [OR 0.041(CI 95%: 0.002– 0.879), p < 0.05].

In a landmark study including 565 women followed for 12-month, the rate of objectively-assessed success was 80.8% in the retropubic-sling group and 77.7% in the TOT group (3.0 percentage-point difference; 95% [CI], − 3.6 to 9.6)²³. In our study, 151 patients had a success rate of 89.1% in the retropubic-sling group and 76.6% in the TOT group [OR :0.045 (CI 95%: 0.004 – 0.464), p < 0.01].

Data on statistical models that predict the outcome of MUS are scarce. Through LR, four prediction models were applied to the Trial of Mid-urethral Slings (TOMUS) data set, to predict bothersome SUI, positive stress test, bothersome UUI, and adverse events within 12 months. The accuracy of these models reached 73% for discrimination between women who will or will not develop UI and 66% for prediction of adverse events^24,25. In our study, success was defined as absence of SUI during follow up. UUI was also evaluated and treated but not considered as failure. Our prediction model based on LR achieved an overall accuracy of 90.7%.

Our ANN is a back-propagation, feed forward type. Similar models were used in urology to diagnose and predict the prognosis of prostatic cancer^26,27 achieving a positive predictive value of 94% for survival, and to form a reliable diagnostic tool based on symptoms and objective measurements²⁸. However, the inclusion of subjective criteria might negatively affect the ability of the model to accurately predict failure and in future studies, a composite objective measured could be used. Besides, ANN has 2 main limitations: first its black-box nature and second it behaves better in larger sample size. As a machine learning tool for classification, SVM has gained popularity. It is straightforward and is considered as one type of data mining which is defined as obtaining valuable information from sizable data sets^29,30. SVM has the unique ability to perform linear classification using non-linear data; the so called “kernel trick”³¹.

However, SVM shortcomings include its sensitivity to parameter selection and high algorithmic complexity.

The wide application of AI models in the prediction of the outcome of sling surgery in women with SUI provides an efficient tool to counsel women prior to this surgery and give as best accurate prediction as we can get.

Also, SVM was used in oncology to predict 5-year overall survival after radical cystectomy³², post-cystectomy recurrence and survival³³ and to differentiate angiomyolipoma from renal cell carcinoma based on texture analysis of CT images with 93.9% accuracy and 87.8% sensitivity³⁴.

Conclusion: Building a statistical model regarding the outcome of MUS surgery with high accuracy and sensitivity is applicable through LR.

With an overall accuracy of 90.7% yet positive predictive value of 61.5%, LR model is far from optimum.

The use of AI is a better alternative to obtain valuable prediction of outcome of sling surgery. Our result using ANN was not much better than LR. Overall accuracy was 90.1% and the discrimination of failed cases was poor. SVM provided higher overall accuracy and predictive value of 93% and 96% respectively, mostly because of its kernel trick principle.

It is evident that using SVM, based on our results, is a way forward to better predict the outcome of sling surgery in the future.

Either one, a larger sample size is needed to obtain better prediction. We plan to further include more cases to our SVM model so that we can improve predictive outcome.

AI:

Artificial intelligence

ALPP:

Abdominal leak point pressure

ANN:

Artificial neural network

BMI:

Body mass index

DM:

Diabetes Mellitus

LR:

Logistic regression

PVR:

Post voiding residual urine

PVS:

Pubovaginal sling

SUI:

Stress urinary incontinence

SVM:

Support vector machine

TOT:

Transobturator tape

TVT:

Tension free vaginal tape

UDS:

Urodynamics

1. Ebbesen, M. H. et al. Prevalence, incidence and remission of urinary incontinence in women: Longitudinal data from the Norwegian HUNT study (EPINCONT). BMC urol. 13(1), 1–10 (2013).

   Google Scholar 

2. MacLennan, A. H. et al. The prevalence of pelvic floor disorders and their relationship to gender, age, parity and mode of delivery. BJOG 12, 1460–1470 (2000).

   Google Scholar 

3. Minassian, V. A., Stewart, W. F. & Wood, G. C. Urinary incontinence in women: Variation in prevalence estimates and risk factors. Obst. Gynecol. 111(2), 324–331 (2008).

   Google Scholar 

4. Melville, J. L. et al. Urinary incontinence in US women: A population-based study. Arch. Int. Med. 165(5), 537–542 (2005).

   Google Scholar 

5. Ulmsten, U. et al. An ambulatory surgical procedure under local anesthesia for treatment of female urinary incontinence. Int. Urogynecol. J. 7, 81–86 (1996).

   CAS  Google Scholar 

6. Albo, M. E. et al. Treatment success of retropubic and transobturator mid urethral slings at 24 months. J. Urol. 188(6), 2281–2287 (2012).

   PubMed  PubMed Central  Google Scholar 

7. Wadie, B. S. & Elhefnawy, A. S. TVT versus TOT, 2-year prospective randomized study. WJU. 31(3), 645–649 (2013).

   Google Scholar 

8. Novara, G. et al. Complication rates of tension-free midurethral slings in the treatment of female stress urinary incontinence: A systematic review and meta-analysis of randomized controlled trials comparing tension-free midurethral tapes to other surgical procedures and different devices. Euro. Urol. 53(2), 288–309 (2008).

   Google Scholar 

9. Dreiseitl, S. & Ohno-Machado, L. Logistic regression and artificial neural network classification models: A methodology review. J. Biomed. Inform. 35(5), 352–359 (2002).

   PubMed  Google Scholar 

10. Tu, J. V. Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J. Clin. Epidemiol. 49(11), 1225–1231 (1996).

    CAS  PubMed  Google Scholar 

11. Vapnik VN, Vapnik VN.: Introduction: Four periods in the research of the learning problem. The nature of statistical learning theory. 1–15, (2000).

12. Cristianini, N. & Shawe-Taylor, J. An introduction to support vector machines and other kernel-based learning methods (Cambridge University Press, 2000).

    Google Scholar 

13. Schölkopf, B., Smola, A. J. & Bach, F. Learning with kernels: support vector machines, regularization, optimization, and beyond (MIT press, 2002).

    Google Scholar 

14. Nilsson, C. G., Palva, K., Rezapour, M. & Falconer, C. Eleven years prospective follow-up of the tension-free vaginal tape procedure for treatment of stress urinary incontinence. Int. Urogynecol. J. 19, 1043–1047 (2008).

    CAS  Google Scholar 

15. Cho, S. T., Song, H. C., Song, H. J., Lee, Y. G. & Kim, K. K. Predictors of postoperative voiding dysfunction following transobsturator sling procedures in patients with stress urinary incontinence. Int. Neurourol. J. 14, 26–33 (2010).

    PubMed  PubMed Central  Google Scholar 

16. Song, P. H., Kwon, D. H., Ko, Y. H. & Jung, H. C. The Long-Term Outcomes of the Tension-free Vaginal Tape Procedure for Treatment of Female Stress Urinary Incontinence: Data from Minimum 13 Years of Follow-Up. Low Urin. Tract. Symptoms 9(1), 10–14 (2017).

    PubMed  Google Scholar 

17. McGuire, E. J. et al. Clinical assessment of urethral sphincter function. J. Urol. 150(5), 1452–1454 (1993).

    CAS  PubMed  Google Scholar 

18. Hosmer, D. W. Jr., Lemeshow, S. & Sturdivant, R. X. Applied logistic regression (John Wiley & Sons, 2013).

    Google Scholar 

19. Bach, F., Hill, S. & Toozs-Hobson, P. The effect of body mass index on retropubic midurethral slings. Am. J. Obstet. Gynecol. 220(4), 371-e1 (2019).

    Google Scholar 

20. Lee, K. S. et al. The long term (5-years) objective TVT success rate does not depend on predictive factors at multivariate analysis: A multicentre retrospective study. Eur. Urol. 53(1), 176–82 (2008).

    PubMed  Google Scholar 

21. Salhi, Y. et al. Predictive factors of immediate post-operative acute urinary retention or voiding dysfunction following mid-urethral sling surgery: A literature review. Prog. Urol. 30(17), 1118–25 (2020).

    CAS  PubMed  Google Scholar 

22. Nager, C. W. et al. Baseline urodynamic predictors of treatment failure 1 year after mid urethral sling surgery. J. Urol. 186(2), 597–603 (2011).

    PubMed  Google Scholar 

23. Richter, H. E. et al. Retropubic versus Transobturator Midurethral Slings for Stress Incontinence. NEJM 362(22), 2066–76 (2010).

    CAS  PubMed  Google Scholar 

24. Urinary Incontinence Treatment Network: The trial of mid-urethral slings (TOMUS): design and methodology. J Appl Res. 8(1) (2008).

25. Jelovsek, J. E. et al. Predicting risk of urinary incontinence and adverse events after midurethral sling surgery in women. Obstet. Gynecol. 127(2), 330–340 (2016).

    PubMed  Google Scholar 

26. Snow, P. B., Smith, D. S. & Catalona, W. J. Artificial neural networks in the diagnosis and prognosis of prostate cancer: A pilot study. J. Urol. 152(5), 1923–6 (1994).

    CAS  PubMed  Google Scholar 

27. Tamey, T. A. et al. Effectiveness of ProstAsure in detecting prostate cancer (PCa) and benign prostatic hyperplasia (BPH) in men age 50 and older. J. Urol. 155(Suppl), 436A (1996).

    Google Scholar 

28. Wadie, B. S., Badawi, A. M. & Ghoneim, M. A. The relationship of the international prostate symptom sscore and objective parameters for diagnosing bladder outlet obstruction Part II the potential usefulness of artificial neural network. J. Urol. 165(1), 35–7 (2001).

    CAS  PubMed  Google Scholar 

29. Wang L.: Support vector machines: theory and applications: Springer Science & Business Media. (2005).

30. Kecman, V. Learning and soft computing: support vector machines, neural networks, and fuzzy logic models (MIT press, 2001).

    Google Scholar 

31. Fu, Y. et al. Exploiting the kernel trick to correlate fragment ions for peptide identification via tandem mass spectrometry. Bioinformatics 20(12), 1948–54 (2004).

    CAS  PubMed  Google Scholar 

32. Wang, G. et al. Output based transfer learning with least squares support vector machine and its application in bladder cancer prognosis. Neurocomputing 387, 279–92 (2020).

    Google Scholar 

33. Hasnain, Z. et al. Machine learning models for predicting post-cystectomy recurrence and survival in bladder cancer patients. PLoS ONE 14(2), e0210976 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

34. Feng, Z. et al. Machine learning-based quantitative texture analysis of CT images of small renal masses: differentiation of angiomyolipoma without visible fat from renal cell carcinoma. Eur. Rad. 28, 1625–33 (2018).

    Google Scholar 

Download references

Authors and Affiliations

1. Urology and Nephrology Center, Mansoura University, Mansoura, Egypt

   Bassem S. Wadie, Ahmed Abdelrasheed & Mohammed Taha

2. Biomedical Engineering, Faculty of Engineering, Cairo University, Giza, Egypt

   Ahmed Abdelrahman, Bassam Mohamed, Alaa Saber & Ahmed Badawi

Contributions

B S Wadie: Designed the study, performed surgery and revised the manuscript A Abdelrasheed: Collected the data and drafted the manuscript. M Taha: Performed statistical analysis A Abdelrahman: Performed the ANN testing. B Mohamed: Shared in ANN development. A Saber: Performed SVM construction. A Badawi: Oversaw the development of ANN and SVM models.

Corresponding author

Correspondence to Bassem S. Wadie.

Competing interests

The authors declare no competing interests.

Ethical approval

This study had written informed consents completed before surgery from all patients . The Urology department board of the hospital approved the research protocol prior to beginning the study. No further IRB approval was sought, because the study was non-interventional and retrospective.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions