Abstract
Clinical trials are important tools to improve knowledge about the effectiveness of new treatments for all diseases, including cancers. However, studies show that fewer than 5% of cancer patients are enrolled in any type of research study or clinical trial. Although there is a wide variety of reasons for the low participation rate, we address this issue by designing a chatbot to help users determine their eligibility via interactive, two-way communication. The chatbot is supported by a user-centered classifier that uses an active deep learning approach to separate complex eligibility criteria into questions that can be easily answered by users and information that requires verification by their doctors. We collected all the available clinical trial eligibility criteria from the National Cancer Institute's website to evaluate the chatbot and the classifier. Experimental results show that the active deep learning classifier outperforms the baseline k-nearest neighbor method. In addition, an in-person experiment was conducted to evaluate the effectiveness of the chatbot. The results indicate that the participants who used the chatbot achieved better understanding about eligibility than those who used only the website. Furthermore, interfaces with chatbots were rated significantly better in terms of perceived usability, interactivity, and dialogue.
- M. Amith, Z. H. U. Anna, R. Cunningham, L. I. N. Rebecca, L. Savas, S. H. A. Y. Laura, and T. A. O. Cui. 2019. Early usability assessment of a conversational agent for HPV vaccination. Stud. Health Technol. Inform. 257 (2019), 17--23.Google Scholar
- R. F. Azevedo, D. Morrow, J. Graumlich, A. Willemsen-Dunlap, M. Hasegawa-Johnson, T. S. Huang, and D. J. Halpin. 2018. Using conversational agents to explain medication instructions to older adults. In Proceedings of the AMIA Annual Symposium. 185. American Medical Informatics Association.Google Scholar
- C. S. Bennette, S. D. Ramsey, C. L. McDermott, J. J. Carlson, A. Basu, and D. L. Veenstra. 2016. Predicting low accrual in the national cancer institute's cooperative group clinical trials. J. Nat. Cancer Inst. 108, 2 (2016).Google Scholar
- T. Bickmore, H. Trinh, R. Asadi, and S. Olafsson. 2018. Safety first: Conversational agents for health care. In Studies in Conversational UX Design, 33--57. Springer, Cham.Google Scholar
- P. P. Breitfeld, M. Weisburd, J. M. Overhage, G. Sledge Jr, and W. M. Tierney. 1999. Pilot study of a point-of-use decision support tool for cancer clinical trials eligibility. J. Amer. Med. Inf. Assoc. 6, 6 (1999), 466--477.Google Scholar
- R. W. Carlson, S. W. Tu, N. M. Lane, T. L. Lai, C. A. Kemper, M. A. Musen, and E. H. Shortliffe. 1995. Computer-based screening of patients with HIV/AIDS for clinical-trial eligibility. Online J. Curr. Clin. Trials 4, 179 (1995).Google Scholar
- C. H. Chuan. 2018. Classifying eligibility criteria in clinical trials using active deep learning. In Proceedings of the 17th IEEE International Conference on Machine Learning and Applications. 305--310.Google Scholar
- M. Cuggia, P. Besana, and D. Glasspool. 2011. Comparing semi-automatic systems for recruitment of patients to clinical trials. Int. J. Med. Inf. 80, 6 (2011), 371--388.Google Scholar
Cross Ref
- E. Fink, L. O. Hall, D. B. Goldgof, B. D. Goswami, M. Boonstra, and J. P. Krischer. 2003. Experiments on the automated selection of patients for clinical trials. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics. 4541--4545.Google Scholar
- E. Fink, P. K. Kokku, S. Nikiforou, L. O. Hall, D. B. Goldgof, and J. P. Krischer. 2004. Selection of patients for clinical trials: An interactive web-based system. Artif. Intell. Med. 31, 241--254.Google Scholar
Digital Library
- A. Ho, J. Hancock, and A. S. Miner. 2018. Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. J. Commun. 68, 4 (2018), 712--733.Google Scholar
Cross Ref
- F. Köpcke, D. Lubgan, R. Fietkau, A. Scholler, C. Nau, M. Stürzl, and D. Toddenroth. 2013. Evaluating predictive modeling algorithms to assess patient eligibility for clinical trials from routine data. BMC Med. Inf. Dec. Making 13, 1 (2013), 134.Google Scholar
- M. Kusner, Y. Sun, N. Kolkin, and K. Weinberger. 2015. From word embeddings to document distances. In Proceedings of the International Conference on Machine Learning. 957--966.Google Scholar
- L. Laranjo, A. G. Dunn, H. L. Tong, A. B. Kocaballi, J. Chen, R. Bashir, D. Surian, B. Gallego, F. Magrabi, A. Y. Lau, and E. Coiera. 2018. Conversational agents in healthcare: A systematic review. J. Amer. Med. Inf. Assoc. 25, 9 (2018), 1248--1258.Google Scholar
Cross Ref
- J. Lilleberg, Y. Zhu, and Y. Zhang. 2015. Support vector machines and word2vec for text classification with semantic features. In Proceedings of the IEEE 14th International Conference on Cognitive Informatics and Cognitive Computing. 136--140.Google Scholar
- Z. Luo, R. Duffy, S. Johnson, and C. Weng. 2010. Corpus-based approach to creating a semantic lexicon for clinical research eligibility criteria from UMLS. In Proceedings of the AMIA Summit on Clinical Research Informatics. 26--31.Google Scholar
- Z. Luo, M. Yetisgen-Yildiz, and C. Weng. 2011. Dynamic categorization of clinical research eligibility criteria by hierarchical clustering. J. Biomed. Inf. 44, 6 (2011), 927--935.Google Scholar
Digital Library
- L. V. D. Maaten and G. Hinton. 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9 (2008), 2579--2605.Google Scholar
- J. M. Metz, C. Coyle, C. Hudson, and M. Hampshire. 2005. An internet-based cancer clinical trials matching resource. J. Med. Internet Res. 7, 3 (2005), e24.Google Scholar
- T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. 2013. Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Proc. Syst. 3111--3119.Google Scholar
- K. Milian, A. Bucur, and A. Ten Teije. 2012. Formalization of clinical trial eligibility criteria: Evaluation of a pattern-based approach. In Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine. 1--4.Google Scholar
- K. Milian, A. Ten Teije, A. Bucur, and F. Van Harmelen. 2011. Patterns of clinical trial eligibility criteria. In Proceedings of the International Workshop on Knowledge Representation for Health Care. Springer, Berlin, 145--157.Google Scholar
- R. Miotto and C. Weng. 2015. Case-based reasoning using electronic health records efficiently identifies eligible patients for clinical trials. J. Amer. Med. Inf. Assoc. 22, e1 (2015), 141--150.Google Scholar
Cross Ref
- J. L. Z. Montenegro, C. A. da Costa, and R. da Rosa Righi. 2019. Survey of conversational agents in health. Exp. Syst. Applic. 129 (2019), 56--67.Google Scholar
Cross Ref
- V. H. Murthy, H. M. Krumbolz, and C. P. Gross. 2004. Participation in cancer clinical trials: Race-, sex-, and age-based disparities. J. Amer. Med. Assoc. 291 (2004), 2720--2726.Google Scholar
- J. Niland, D. Dorr, G. El Saadawi, P. Embi, R. L. Richesson et al. 2007. Knowledge representation of eligibility criteria in clinical trials. In Proceedings of the American Medical Informatics Association Annual Symposium.Google Scholar
- M. Peleg, S. Tu, J. Bury, P. Ciccarese, J. Fox et al. 2003. Comparing computer-interpretable guideline models: A case-study approach. J. Amer. Med. Inf. Assoc. 10, (2003) 52--68.Google Scholar
- D. Rubin, J. Gennari, S. Srinivas, A. Yuen, H. Kaizer, M. Musen, et al. 1999. Tool support for authoring eligibility criteria for cancer trials. In Proceedings of the AMIA Symposium. 369--373.Google Scholar
- I. Sim, B. Olasov, and S. Carini. 2004. An ontology of randomized controlled trials for evidence-based practice: Content specification and evaluation using the competency decomposition method. J. Biomed. Inf. 37 (2004), 108--119.Google Scholar
Digital Library
- D. Tang, F. Wei, N. Yang, M. Zhou, T. Liu, and B. Qin. 2014. Learning sentiment-specific word embedding for Twitter sentiment classification. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics 1 (2014), 1555--1565.Google Scholar
- S. W. Tu, M. Peleg, S. Carini, M. Bobak, J. Ross, D. Rubin, and I. Sim. 2011. A practical method for transforming free-text eligibility criteria into computable criteria. J. Biomed. Inf. 44, 2 (2011), 239--250.Google Scholar
Digital Library
- D. Utami, B. Barry, T. Bickmore, and M. Paasche-Orlow. 2013. A conversational agent-based clinical trial search engine. In Proceedings of the Annual Symposium on Human-Computer Interaction and Information Retrieval (HCIR’13).Google Scholar
- R. Wallace. 2003. The Elements of AIML Style. Alice AI Foundation.Google Scholar
- Z. Zhang, T. W. Bickmore, and M. K. Paasche-Orlow. 2017. Perceived organizational affiliation and its effects on patient trust: Role modeling with embodied conversational agents. Patient Educ. Counsel. 100, 9 (2017), 1730--1737.Google Scholar
Cross Ref
- S. Zhang, F. Liang, W. Li, and I. Tannock. 2016. Comparison of eligibility criteria between protocols, registries, and publications of cancer clinical trials. JNCI: J. Nat. Cancer Ins. 108 (2016), 11.Google Scholar
Index Terms
Creating and Evaluating Chatbots as Eligibility Assistants for Clinical Trials: An Active Deep Learning Approach towards User-centered Classification
Recommendations
Inferring appropriate eligibility criteria in clinical trial protocols without labeled data
DTMBIO '12: Proceedings of the ACM sixth international workshop on Data and text mining in biomedical informaticsWe consider the user task of designing clinical trial protocols and propose a method that outputs the most appropriate eligibility criteria from a potentially huge set of candidates. Each document d in our collection D is a clinical trial protocol which ...
A semantic framework for intelligent matchmaking for clinical trial eligibility criteria
Survey papers, special sections on the semantic adaptive social web, intelligent systems for health informatics, regular papersAn integral step in the discovery of new treatments for medical conditions is the matching of potential subjects with appropriate clinical trials. Eligibility criteria for clinical trials are typically specified as inclusion and exclusion criteria for ...
Patterns of clinical trial eligibility criteria
KR4HC'11: Proceedings of the 3rd international conference on Knowledge Representation for Health-CareMedical research would benefit from automatic methods that support eligibility evaluation for patient enrollment in clinical trials and design of eligibility criteria. In this study we addressed the problem of formalizing eligibility criteria. By ...






Comments