A Case-Based Approach for Unravelling the Complexity in Adoption Decision-Making in Enterprise Health Information Systems

Health technology assessment (HTA) is crucial in making adoption decisions for emerging health technologies, such as an enterprise health information system. Variance-based approaches are commonly used in HTA. However, given health technologies’ complexity and multifaceted nature, relying solely on a net effect approach could be misleading. Instead, case-based approaches such as qualitative comparative analysis (QCA) have the uniqueness to capture combinations of complex factors that align with the outcomes being studied. This study aims to demonstrate using QCA as a case-based approach to provide additional nuances in traditional evidence-based synthesis in HTA. We designed a measurement model based on technology acceptance and information systems success models from literature to collect clinicians’ experience with an electronic medication management system (EMMS). Confirmatory factor analysis (CFA) was conducted to examine the dimension's reliability and validity in the measurement model. QCA was then performed and revealed three different configurations that led to the successful adoption of EMMS for doctors and one for nurses. Information quality, perceived usefulness, service quality and satisfaction were core conditions indispensable to EMMS adoption and success. Doctors and nurses have interrelated but different results. Overall, we demonstrated that QCA, as a case-based approach, can add valuable information to HTA and adoption decision-making about new health technologies.


INTRODUCTION
Nowadays, health practitioners operate in complicated situations riddled with interruptions, time constraints, and an overload of information [1].The use of health technologies is substantially increasing.Clinicians must swiftly assimilate large volumes of data to make decisions in volatile and complicated environments [2].However, healthcare delivery is often complex, with many different processes and workflows involved.It's often challenging to comprise various data types that need to be analysed and reported in a meaningful way to support decision-making.Interconnections between different clinicians' usages make it challenging to evaluate a health technology's effectiveness.Thus, a comprehensive health technology assessment (HTA) is crucial in making adoption decisions for emerging health technologies.
While many studies suggest that different dimensions are likely to have a synergistic impact on health technology success [3][4][5][6][7], most of these studies used variance-based models (VBM) for their analysis.VBM estimates each dimension's impact on health technology use after isolating the influence of other dimensions [8].However, given the complexity and multifaceted nature of health technologies, relying solely on a symmetric and net effect approach could be misleading as no single dimension can independently lead to their success.Hence, HTA requires an approach that can assess the interaction between a wide range of components and processes, and also determine their impacts on the system's performance.
A case-based and configurational approach, such as qualitative comparative analysis (QCA), can provide various advantages in a comprehensive HTA.The approach can provide rigorous modelling and analysis of complex relationships and interactions.It bridges qualitative and quantitative methodologies and computes the degree to which a case belongs to a set [9,10].This addresses the issue of multi-collinearity that always occurs in VBM.Cases opposing the observed net effects are almost always present; not all cases in the data show a negative or positive association between the independent and dependent variables [8,11].Configurational approaches can address this limitation by reporting the outcome as configurations of conditions [12].For instance, QCA is unique in capturing combinations of complex factors that cover or align with the outcomes being studied [12].In addition, case-based approaches provide a more nuanced understanding of the interaction of different levels of outcomes by considering the consistency and coverage of configurations [13].It can tackle the most challenging aspect of HTA, such as the fact that many health technologies have different types of users with different roles, but their usage is highly interactive.No other statistical analysis approaches can provide views from this angle.Such approaches can provide and add valuable information for healthcare facilities to make adoption decisions about new health technologies.Therefore, the aim of this study is to demonstrate the use of QCA, a case-based approach, to provide additional nuances in traditional evidence-based synthesis in HTA.

METHODS
To highlight the application of QCA in HTA, we designed a measurement model based on technology acceptance and information systems success models from literature to collect clinicians' experience with an electronic medication management system (EMMS) after one year of implementation.EMMS is a type of enterprise health information system (HIS) that supports medication management processes to enhance the accuracy and accessibility of medication information among health professionals [14,15].

D&M IS Success Model
The updated DeLone and McLean Information Systems Success Model (D&M IS Success Model) was selected as our theoretical framework [16].The model includes six dimensions: Information Quality (IQ), System Quality, Service Quality (SQ), Use/Intention to Use, User satisfaction (USAT), and Net Benefit.Many studies review that different dimensions are likely to have a synergistic effect to determine the success of HIS [3,5,7].While the D&M IS Success Model has the advantage of addressing the multidimensional nature of HIS and is commonly recognised and well-validated [17], the model must be contextualised to include clinical aspects associated with the healthcare and EMMS domain.Given the mandatory nature of the EMMS use, we focus on the net benefits and disregard the "Use/Intention to Use" dimension.This study approached the "net benefit" definition by considering the overall medication management benefits provided by EMMS, namely, Overall Benefit (OB).Also, system quality was measured in perceived usefulness (PU) and perceived ease of use (PEoU) in this study.

Conceptual model
Due to the context of EMMS, we modified the updated D&M IS Success Model (Figure 1), and constructed a conceptual model for case-based data analysis (Figure 2).This conceptual model aims to capture insights from the modified IS Success Model as a Venn diagram, which shows the set relationships between the conditions.Each condition is considered a set and denoted as a circle.Cases within the set are inside the circle, meaning the condition is present.The overlap of these conditions represents the interactions between different conditions.The outcome set, clinicians with a high level of OB, is also denoted as a circle.Using QCA, this model enables us to investigate the combinations of conditions (IQ, PEoU, PU, SQ, USAT) that lead to a high level of OB.The sufficient configurations of conditions can be determined by investigating how the outcome set would overlap with the condition sets.

Survey Design
We selected EMMS as the health technology to assess due to its inherent complexity and the need for a nuanced evaluation approach.The conceptual model was examined using survey data collected from a hospital located in a health service district that provides care to more than two million people, and it was one of the first few hospitals that implemented the EMMS across NSW.The survey data was collected a year after the implementation.We explicitly conducted the survey study during this time as users were more comfortable with using EMMS after a year and would likely provide more accurate and dependable feedback.In addition, EMMS is the most recently implemented large-scale enterprise HIS in NSW, making it a meaningful example.
Table 1 shows the questionnaire items included to measure six latent dimensions in the conceptual model.There were 201 responses collected, including 96 doctors and 95 nurses.During the medication management process, the major roles and tasks between different users are different but can also overlap [15].For instance, doctors (the prescribers) have more responsibilities in prescribing the medication, while they also have responsibilities in reconciling the patient's current medication chart by reviewing their medication history.Nurses are mostly responsible for the administration of medicines to patients.In addition, nurses are responsible for documenting and recording the administration of medicines, which could ensure the continuity of medication management.Hence, considering the differences and the interconnections between different roles in using EMMS is essential.

Confirmatory Factor Analysis (CFA)
A confirmatory factor analysis (CFA) was conducted to examine the dimension's reliability and validity in the measurement model using IBM SPSS Amos.The convergent validity was examined by evaluating each dimension's standardised factor loading, composite reliability (CR), and average variance extracted (AVE).The discriminant validity was lastly assessed by examining the square root of AVE and the correlation between these dimensions.

Data Calibration
The measurement of dimensions was computed by averaging their allocated questionnaire items.The 7-point Likert scale data was then directly calibrated into fuzzy set values ranging from 0 to 1.The three anchors selected were 6 for full set inclusion, 4.5 for cross-over point, and 3.5 for full set exclusion.A score of 6 is strong enough to classify that the participant quite agreed or strongly agreed with the measurement items.A score below 3.5 indicates that the participant qualitatively disagrees with the measurement items.A score of 4.5 is precisely in the centre of the membership, meaning that it could be a member or non-member of the set.

Data Analysis
The data analysis was performed using the software fsQCA 4.1 Mac.Firstly, conditions having a consistency threshold of at least 0.90 [13], a coverage score greater than 0.60 [18], and a Relevance of Necessity (RoN) score greater than 0.60 [18] were considered as necessary conditions.Secondly, the calibrated data was incorporated into a truth table listing all possible configurations of conditions.Each condition could be high-level or low-level [18].The truth table only represented conditions with the values 0 and 1, where 0 denoted the absence of the condition and 1 denoted its presence.Thirdly, a sufficiency analysis was performed.The initial truth tables were reduced by providing the frequency threshold of 3. The outcome in the truth table for each possible configuration could then be defined as absence (0) or presence (1), based on whether their raw consistency scores meet the threshold of 0.80.This identified any sufficient configurations, meaning that whenever the outcome occurs, these configurations are present as well.Finally, logical minimisation was performed to determine the simplest possible configurations associated with the outcomes using the Quine-McCluskey algorithm [19].Core and peripheral conditions, as well as raw and unique coverage, were determined for each solution.Core conditions have a stronger causal relationship with the outcome than peripheral conditions.Raw coverage indicates "which share of the outcome is explained by a certain alternative path", while unique coverage indicates "which share of the outcome is exclusively explained by a certain alternative path" [20].

Reliability and Validity
Table 2 shows the result of CFA.Standardised factor loadings for each questionnaire item to the dimension were all greater than 0.7.All CR values were greater than 0.7 and all AVE value were over 0.5.Convergent validity was confirmed.For all dimensions, their square roots of AVE were greater than their correlations with other dimensions.Most of the dimension correlation did not exceed 0.8.The discriminant validity was also confirmed.Thus, the CFA confirmed the reliability and validity of each dimension in the measurement model.

Necessity Analysis Results
Table 3 displays the necessity analysis results.IQ is the necessary condition for doctors to achieve a high OB level.All conditions are necessary for nurses.IQ, PU, and USAT are necessary for all clinicians.

Logical Minimisation Results
The logical minimisation results are presented in Table 4.
Configurations 1.1 suggests that doctors who considered the EMMS easy to use, and the hospital's services/support to be of high After the implementation, I find the information provided by EMMS accurate.2 After the implementation, I find the information provided by EMMS relevant.3 After the implementation, I find the information provided by EMMS timely.4 After the implementation, I find the information provided by EMMS complete.5 After the implementation, I find the information provided by EMMS appropriate in amount.6 After the implementation, I find the information provided by EMMS consistent and concise in format.7 After the implementation, I find the information provided by EMMS accessible.Perceived Ease of Use (PEoU) My interaction with the EMMS system is clear and understandable.Overall, using EMMS reduces medication errors and adverse drug events.2 Overall, using EMMS improves compliance with guidelines, protocols and standards.3 Overall, using EMMS improves quality of medication management.4 Overall, using EMMS improves safety of medication management.5 Overall, using EMMS improves effectiveness of medication management.
Note: These items were measured using a 7-point Likert scale.If the participant selected 1, they strongly disagreed with the statement.If the participant selected 7, they strongly agreed with the statement.
quality, would perceive a high level of OB from the EMMS.Configurations 1.2 suggests that doctors who considered the information provided by EMMS to be of high quality, useful and satisfying to use would perceive a high level of OB.Configurations 1.3 suggests that doctors who considered the information offered by the EMMS to be of high quality, easy to use, satisfying to use, and the hospital's services/support to be of high quality would perceive a high level of OB from the EMMS.Core conditions for these configurations are IQ, PU, SQ, and USAT.The overall consistency for the doctors' results was 0.939, indicating that 93.9% of the doctors characterised by these configurations were among those with a high OB level.The overall coverage for the doctors' results was 0.862, indicating that the solution could explain 86.2% of the doctors with a high OB level.Configuration 2 suggests that nurses, who considered the information offered by the EMMS to be of high quality, easy to use, useful, and satisfying to use, would perceive a high level of OB from the EMMS.The overall consistency for the nurses' result was 0.970, indicating that 97.0% of the nurses characterised by Configurations 2 were among those with a high OB level.The overall coverage for the nurses' result was 0.894, indicating that the solution could explain 89.4 % of the nurses with a high OB level.Configurations 3 and 4 suggest that the absence of these conditions would lead to adoption failure for both doctors and nurses.

DISCUSSION
The use of case-based approach such as QCA shows nuance presentation and understanding of the user experience.It appears that satisfaction was a core condition for all configurations except Configurations 1.1 and 3.1.This suggests that clinician satisfaction is an indispensable factor for the configurations that lead to EMMS success.Ensuring clinician satisfaction in all usage levels is essential to EMMS success.This also illustrates that the QCA approach can capture the mediation effect of the USAT dimension in the Modified IS Success Model.Doctors and nurses have entirely different patterns in their configurations, leading to high OB levels.For instance, doctors value service quality, but nurses value whether the system is easy to use.These differences in their configurations toward obtaining benefits from the EMMS illustrate their different roles in the medication management process.QCA allows us to visualise the interaction of different levels of outcomes between different types of users.This allows the evaluation to propose several strategies for improving the overall or user-specific outcome.Furthermore, comparing the raw coverage value between configurations for doctors, configuration 1.2 had the highest raw coverage of 0. 768.This indicates that configuration 1.2 was the most relevant configuration to explain the presence of high OB levels in doctors' use of the EMMS.Overall, this application example proposes that no single factor in the Modified IS Success Model could directly lead to the system's success.More specifically, this study elucidates IQ, From a HTA perspective, this study developed an understanding of clinicians' benefits gained when using the EMMS.These understandings extend the existing literature on HTA, specifically about the EMMS implementation in Australia, which had very limited literature.But more importantly, this study illustrates how casebased approach such as QCA allows medical informatics experts to draw insights from an entirely new angle.This study demonstrates creativity and innovation in the current field of evidence-based decision-making.Health organisations may use this approach to identify the most relevant combinations of conditions that explain the outcome and conduct a configuration-specific improvement for targeted users or even individuals.Thus, they can improve the health technology efficiently, and minimising the time required for an upgrade, which is crucial in healthcare settings.Such case-based analytic approaches can reveal the complexity for adoption decisionmaking, allowing improved clinician and patient outcomes.
In conclusion, in this study, we introduced an innovative approach to perform HTA, in which we illustrated why case-based approach is ideal for such assessment.The EMMS implemented in an Australian hospital was comprehensively evaluated using a survey study based on the Modified IS Success Model using QCA.The study considered the overall benefit gained by the clinicians as the ultimate outcome and elucidated multiple configurations that led to the EMMS success.

Figure 1 :
Figure 1: Modified IS Success Model

Table 1 :
Measurement Model (Questionnaire Items Used)

Table 3 :
Necessity Analysis Results

Table 4 :
Multiple Paths to EMMS Success Black circles (G) indicate the presence of a condition.The circles with "X" (⊗) indicate the absence of the condition.Core conditions were highlighted with a larger circle, and peripheral conditions were indicated with a smaller circle.Conditions that did not contribute to the outcome were left blank in the table.