skip to main content
research-article

Predicting the Evolution of Pain Relief: Ensemble Learning by Diversifying Model Explanations

Authors Info & Claims
Published:14 September 2021Publication History
Skip Abstract Section

Abstract

Modeling from data usually has two distinct facets: building sound explanatory models or creating powerful predictive models for a system or phenomenon. Most of recent literature does not exploit the relationship between explanation and prediction while learning models from data. Recent algorithms are not taking advantage of the fact that many phenomena are actually defined by diverse sub-populations and local structures, and thus there are many possible predictive models providing contrasting interpretations or competing explanations for the same phenomenon. In this article, we propose to explore a complementary link between explanation and prediction. Our main intuition is that models having their decisions explained by the same factors are likely to perform better predictions for data points within the same local structures. We evaluate our methodology to model the evolution of pain relief in patients suffering from chronic pain under usual guideline-based treatment. The ensembles generated using our framework are compared with all-in-one approaches of robust algorithms to high-dimensional data, such as Random Forests and XGBoost. Chronic pain can be primary or secondary to diseases. Its symptomatology can be classified as nociceptive, nociplastic, or neuropathic, and is generally associated with many different causal structures, challenging typical modeling methodologies. Our data includes 631 patients receiving pain treatment. We considered 338 features providing information about pain sensation, socioeconomic status, and prescribed treatments. Our goal is to predict, using data from the first consultation only, if the patient will be successful in treatment for chronic pain relief. As a result of this work, we were able to build ensembles that are able to consistently improve performance by up to 33% when compared to models trained using all the available features. We also obtained relevant gains in interpretability, with resulting ensembles using only 15% of the total number of features. We show we can effectively generate ensembles from competing explanations, promoting diversity in ensemble learning and leading to significant gains in accuracy by enforcing a stable scenario in which models that are dissimilar in terms of their predictions are also dissimilar in terms of their explanation factors.

References

  1. M. Abad-Grau, J. Ierache, C. Cervino, and P. Sebastiani. 2008. Evolution and challenges in the design of computational systems for triage assistance. J. Biomed. Inform. 41, 3 (2008), 432–441. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. R. Agrawal, J. Gehrke, D. Gunopulos, and P. Raghavan. 2005. Automatic subspace clustering of high dimensional data. Data Mining Knowl. Discov. 11, 1 (2005), 5–33. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. J. M. Benitez, J. L. Castro, and I. Requena. 1997. Are artificial neural networks black boxes?IEEE Trans. Neural Netw. 8, 5 (Sept. 1997), 1156–1164. DOI:https://doi.org/10.1109/72.623216 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. L. Bernardes, M. Carvalho, S. Harnik, M. Teixeira, J. Ottolia, D. Castro, A. Veloso, R. Francisco, C. Listik, R. Galhardoni, V. da Silva, L. Moreira, A. de Amorim Filho, A. Fernandes, and D. Ciampi de Andrade. 2021. Sorting pain out of salience: Assessment of pain facial expressions in the human fetus. Pain Rep. 6, 1 (2021), e882.Google ScholarGoogle Scholar
  5. F. Blyth, L. March, A. Brnabic, L. Jorm, M. Williamson, and M. Cousins. 2001. Chronic pain in Australia: A prevalence study. Pain 89, 2-3 (2001), 127–134. DOI:https://doi.org/10.1016/S0304-3959(00)00355-9Google ScholarGoogle ScholarCross RefCross Ref
  6. L. Breiman. 2001. Random forests. Mach. Learn. 45, 1 (2001), 5–32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. M. Chen, K. Weinberger, and Y. Chen. 2011. Automatic feature decomposition for single view co-training. In International Conference on Machine Learning. 953–960. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. T. Chen and C. Guestrin. 2016. XGBoost: A scalable tree boosting system. In International Conference on Knowledge Discovery and Data Mining. 785–794. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Y. Cheng and G. Church. 2000. Biclustering of expression data. In Proceedings of the 8th International Conference on Intelligent Systems for Molecular Biology. 93–103. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. R. Dworkin, D. Turk, K. Wyrwich, D. Beaton, D. Cleeland, J. Farrar, J. Haythornthwaite, M. Jensen, R. Kerns, D. Ader, N. Brandenburg, L. Burke, D. Cella, J. Chandler, P. Cowan P., R. Dimitrova, R. Dionne, S. Hertz, A. Jadad, N. Katz, H. Kehlet, L. Kramer, D. Manning, C. McCormick, M. McDermott, H. McQuay, S. Patel, L. Porter, S. Quessy, B. Rappaport, C. Rauschkolb, D. Revicki, and M. Rothman, 2008. Interpreting the clinical importance of treatment outcomes in chronic pain clinical trials: IMMPACT recommendations. J. Pain 9, 2 (2008), 105–121.Google ScholarGoogle Scholar
  11. Radwa Elshawi, Youssef Sherif, Mouaz Al-Mallah, and Sherif Sakr. 2019. Interpretability in healthcare a comparative study of local machine learning interpretability techniques. In IEEE 32nd International Symposium on Computer-Based Medical Systems (CBMS’19). IEEE, 275–280.Google ScholarGoogle Scholar
  12. M. Ester, H. Kriegel, J. Sander, and X. Xu. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In International Conference on Knowledge Discovery and Data Mining. 226–231. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. T. Fawcett. 2006. An introduction to ROC analysis. Pattern Recog. Lett. 27, 8 (2006), 861–874. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. K. Ferreira, T. Bastos, D. Ciampi, A. Silva, J. Appolinario, M. Jacobsen, and M. Latorre. 2016. Prevalence of chronic pain in a metropolitan area of a developing country: A population-based study. Arquivos de Neuro-psiquiatria 74, 12 (2016), 990–998.Google ScholarGoogle Scholar
  15. A. Friesen and P. Domingos. 2015. Recursive decomposition for nonconvex optimization. In International Joint Conference on Artificial Intelligence. 253–259. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. B. Goldstein, A. Navar, and R. Carter. 2016. Moving beyond regression techniques in cardiovascular risk prediction: Applying machine learning to address analytic challenges. Eur. Heart J. 38, 23 (2016), 1805–1814.Google ScholarGoogle Scholar
  17. J. Hanley and B. McNeil. 1982. The meaning and use of the area under a receiver operating characteristic ROC curve. Radiology 143 (1982), 29–36.Google ScholarGoogle ScholarCross RefCross Ref
  18. J. Hill, K. Dunn, M. Lewis, R. Mullis, C. Main, N. Foster, and E. Hay. 2008. A primary care back pain screening tool: Identifying patient subgroups for initial treatment. Arthr. Rheum 5, 59 (2008), 632–641.Google ScholarGoogle Scholar
  19. Yan Huang, Huiru Zheng, Chris Nugent, Paul McCullagh, Norman Black, Kevin E. Vowles, and Lance McCracken. 2010. Feature selection and classification in supporting report-based self-management for people with chronic pain. IEEE Trans. Inf. Technol. Biomed. 15, 1 (2010), 54–61. IEEE. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. D. Jha and G. Kwon. 2017. Diagnosis of Alzheimer’s disease using a machine learning technique. Alzh. Dement. 13, 7 (2017), 1538.Google ScholarGoogle Scholar
  21. L. Kuncheva and C. Whitaker. 2003. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 51, 2 (2003), 181–207. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Ludmila I. Kuncheva, Fabio Roli, Gian Luca Marcialis, and Catherine A. Shipp. 2001. Complexity of data subsets generated by the random subspace method: An experimental investigation. In International Workshop on Multiple Classifier Systems. Springer, 349–358. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. S. Lundberg and S. Lee. 2017. A unified approach to interpreting model predictions. In Conference on Neural Information Processing Systems. 4768–4777. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. S. Lundberg, B. Nair, M. Vavilala, M. Horibe, M. Eisses, T. Adams, D. Liston, D. Low, S. Newman, J. Kim, and S. Lee. 2018. Explainable machine learning predictions to help anesthesiologists prevent hypoxemia during surgery. Nat. Biomed. Eng. 2, 10 (2018), 749–760. DOI:https://doi.org/10.1101/206540Google ScholarGoogle ScholarCross RefCross Ref
  25. Scott M. Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M. Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. 2020. From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2, 1 (Jan. 2020), 56–67. DOI:https://doi.org/10.1038/s42256-019-0138-9Google ScholarGoogle ScholarCross RefCross Ref
  26. O. Maimon and L. Rokach. 2002. Improving supervised learning by feature decomposition. In International Symposium on the Foundations of Information and Knowledge Systems. 178–196. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Morteza Mashayekhi and Robin Gras. 2015. Rule extraction from random forest: The RF+HC methods. In Advances in Artificial Intelligence, Denilson Barbosa and Evangelos Milios (Eds.). Vol. 9091. Springer International Publishing, Cham, 223–237. DOI:https://doi.org/10.1007/978-3-319-18356-5_20Google ScholarGoogle Scholar
  28. G. McLachlan and D. Peel. 2000. Finite Mixture Models. Wiley Series in Probability and Statistics. John Wiley & Sons, New York.Google ScholarGoogle Scholar
  29. R. Melzack. 1975. The McGill pain questionnaire. major properties and scoring methods. Pain 1 (1975), 277–299.Google ScholarGoogle ScholarCross RefCross Ref
  30. A. Navani and G. Li. 2016. Chronic pain challenge: A statistical machine-learning method for chronic pain assessment. J. Rec. Adv. Pain 2, 3 (2016), 82–86. DOI:https://doi.org/10.5005/jp-journals-10046-0048Google ScholarGoogle Scholar
  31. E. Nigri, N. Ziviani, F. Cappabianco, A. Antunes, and A. Veloso. 2020. Explainable deep CNNs for MRI-based diagnosis of Alzheimer’s disease. In International Joint Conference on Neural Networks. IEEE, 1–8.Google ScholarGoogle Scholar
  32. A. Oliveira and S. Madeira. 2004. Biclustering algorithms for biological data analysis: A survey. IEEE/ACM Trans. Computat. Biol. Bioinf. 1, 1 (2004), 24–45. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. T. Pansombut, W. Hendrix, Z. Jacob Gao, B. Harrison, and N. Samatova. 2011. Biclustering-driven ensemble of Bayesian belief network classifiers for underdetermined problems. In International Joint Conference on Artificial Intelligence. 1439–1445. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. VanderPlas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12 (2011), 2825–2830. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. A. Pieterse, A. Stiggelbout, and V. Montori. 2019. Shared decision making and the importance of time. JAMA - J. Amer. Med. Assoc. 322, 1 (2019), 25–26. DOI:https://doi.org/10.1001/jama.2019.3785Google ScholarGoogle Scholar
  36. N. Pombo, P. Araújo, and J. Viana. 2014. Knowledge discovery in clinical decision support systems for pain management: A systematic review. Arti. Intell. Med. 60, 1 (2014), 1–11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. M. Ribeiro, S. Singh, and C. Guestrin. 2016. “Why should i trust you?”: Explaining the predictions of any classifier. In International Conference on Knowledge Discovery and Data Mining. 1135–1144. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. M. Ribeiro, S. Singh, and C. Guestrin. 2018. Anchors: High-precision model-agnostic explanations. In AAAI Conference on Artificial Intelligence. 1527–1535.Google ScholarGoogle Scholar
  39. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Anchors: High-precision model-agnostic explanations. In AAAI Conference on Artificial Intelligence, Vol. 18. 1527–1535.Google ScholarGoogle Scholar
  40. Greg Ridgeway, David Madigan, Thomas Richardson, and John O’Kane. 1998. Interpretable boosted naïve Bayes classification. In KDD. 101–104. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Michael E. Robinson, Andrew M. O’Shea, Jason G. Craggs, Donald D. Price, Janelle E. Letzen, and Roland Staud. 2015. Comparison of machine classification algorithms for fibromyalgia: Neuroimages versus self-report. J. Pain 16, 5 (2015), 472–477. Elsevier.Google ScholarGoogle ScholarCross RefCross Ref
  42. G. Shmueli. 2010. To explain or to predict?Statist. Sci. 25, 3 (2010), 289–310.Google ScholarGoogle Scholar
  43. Martin Tamajka, Wanda Benesova, and Matej Kompanek. 2019. Transforming convolutional neural network to an interpretable classifier. In International Conference on Systems, Signals and Image Processing (IWSSIP’19). IEEE, 255–259. ZSCC: 0000002.Google ScholarGoogle Scholar
  44. A. Tanay, R. Sharan, M. Kupiec, and R. Shamir. 2004. Revealing modularity and organization in the yeast molecular network by integrated analysis of highly heterogeneous genomewide data. Proc. Natl. Acad. Sci. 101, 9 (2004), 2981–2986.Google ScholarGoogle ScholarCross RefCross Ref
  45. A. Tanay, R. Sharan, and R. Shamir. 2005. Biclustering algorithms: A survey. Handb. Computat. Molec. Biol. 9 (2005), 26–1.Google ScholarGoogle Scholar
  46. D. Valle, T. Pimentel, and A. Veloso. 2020. Assessing the reliability of visual explanations of deep models with adversarial perturbations. In International Joint Conference on Neural Networks. IEEE, 1–8.Google ScholarGoogle Scholar
  47. L. van der Maaten. 2009. Learning a parametric embedding by preserving local structure. In International Conference on Artificial Intelligence and Statistics. 384–391.Google ScholarGoogle Scholar
  48. T. Vos, A. Flaxman, M. Naghavi, R. Lozano, C. Michaud, M. Ezzati, K. Shibuya, J. Salomon, S. Abdalla, and V. Aboyans. 2012. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990–2010: A systematic analysis for the Global Burden of Disease Study 2010. Lancet 380, 9859 (2012), 2163–2196.Google ScholarGoogle Scholar
  49. J. Ward. 1963. Hierarchical grouping to optimize an objective function. J. Amer. Statist. Assoc. 58 (1963), 236–244.Google ScholarGoogle ScholarCross RefCross Ref
  50. K. White, J. Lee, and A. de C Williams. 2016. Are patients’ and doctors’ accounts of the first specialist consultation for chronic back pain in agreement?J. Pain Res. (2016), 1109–1120.Google ScholarGoogle Scholar
  51. A. Williams and K. Craig. 2016. Updating the definition of pain. PAIN 157 (05 2016), 1. DOI:https://doi.org/10.1097/j.pain.0000000000000613Google ScholarGoogle Scholar

Index Terms

  1. Predicting the Evolution of Pain Relief: Ensemble Learning by Diversifying Model Explanations

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!