Analysing User Experience of Dynamic Group Formation (DGF) in Intelligent Tutor Collaborative Learning (ITSCL) using Aspect-Based Sentiment Analysis

Dynamic group formation is a crucial component of Computer-Supported Collaborative Learning (CSCL). It encourages students to collaborate in flexible, adaptive groups, which can improve learning results and promote collaborative abilities. As a result, this conference publication aims to evaluate the user experience of dynamic group formation in Intelligent Tutoring Supported Collaborative Learning (ITSCL), an intelligent tutoring collaborative learning system. The evaluation is essential for understanding the impact of this feature on user satisfaction and learning outcomes and can inform the design of more effective and user-friendly ITSCL systems. The authors present the user-experience evaluation of dynamic group formation in Intelligent Tutoring Supported Collaborative Learning (ITSCL). We conducted a user-experience experiment using an online questionnaire to gather user feedback on questions related to the user-experience paradigm and then employed an aspect-based sentiment analysis approach to extract user-experience centric aspects and emotions from user comments. The results demonstrate that the dynamic group formation feature of ITSCL positively impacts the user experience, and users reported satisfaction with the system's flexibility, adaptability, and interactivity. Moreover, our analysis provides a deep understanding of the aspects and emotions that are crucial to the user experience and can inform the design of future ITSCL systems.


INTRODUCTION
Collaborative learning is a pedagogical approach emphasising group work and peer interaction to achieve common goals.It encourages students to share ideas, perspectives, and skills while engaging in problem-solving, critical thinking, and decision-making [1].Collaborative learning enhances learning outcomes by fostering a supportive and interactive learning environment, promoting active participation and engagement, and providing opportunities for students to receive and provide feedback [2].Additionally, it helps to develop critical social skills such as communication, teamwork, and leadership, which are crucial for success in today's world.However, effective collaborative learning requires careful planning and management, clear communication, and defined roles to ensure that all students contribute equally to the group's success [3].
Computer-Supported Collaborative Learning (CSCL) is an innovative pedagogical approach that emphasises collaboration and interaction among learners, facilitated by technology.CSCL integration with Intelligent Tutoring System (ITS) can provide personalised and adaptive learning experiences that cater to individual learner needs while promoting social and cognitive development through collaborative activities [2].ITS with CSCL can offer learners opportunities to collaborate in real-time, receive and provide feedback, and participate in group problem-solving activities.This integration can enhance the effectiveness of the learning process by providing learners with a supportive and engaging environment that promotes active participation, critical thinking, and metacognitive awareness [4].The successful implementation of CSCL in ITS requires careful planning and management, considering factors such as group formation, individual differences, and technological support.Overall, using CSCL in ITS represents a promising avenue for research in educational technology and can potentially improve student learning outcomes in various domains [5].Intelligent Tutoring Supported Collaborative Learning (ITSCL) was introduced to incorporate CSCL within ITS to enhance collaborative learning with the help of technological advancement [2].ITSCL is an extension of ITS that enables multiple learners to interact within the system and facilitates three distinct interaction levels, each serving a unique purpose.The first level allows for individual learning through learner-tutor interaction.In contrast, the second and third levels promote collaborative learning through learner-learner and tutor-group interactions with a collective group of learners [2].
The collaborative learning process in Computer-Supported Collaborative Learning using ITS is significantly impacted by Group Formation (GF), which is, in turn, influenced by a range of factors such as diversity in personal characteristics and social, cultural, psychological, and cognitive aspects [6].While various methods of group formation have been developed to address the issue of group compatibility, there is still a need for an optimal solution for dynamic group formation.Furthermore, there needs to be more research examining the combination of collaborative group formation with a collaborative platform.
Concerning the above, in this paper, we evaluate the user experience of the Dynamic Group Formation (DGF) technique using ITSCL, which has been proposed to achieve balanced heterogeneous groups of students based on their learning style and knowledge level [6].The proposed approach has been evaluated in terms of students' performance using the t-test technique and calculating learning gain; however, the user experience of the proposed method in terms of usefulness, usability, accessibility, credibility, and navigation of the system needs evaluation based on user experience principles.
The paper distribution is as follows: Section 2 explains the Dynamic Group Formation (DGF) approach using ITSCL.Section 3 briefly discusses the proposed methodology and methods utilised for user-experience evaluation.Section 4 consists of evaluation results.Finally, Section 5 concludes the study with a discussion and analysis.

DYNAMIC GROUP FORMATION USING INTELLIGENT TUTORING COLLABORATIVE SYSTEM (DGF-ITSCL) 2.1 ITSCL Collaborative Platform
The ITSCL platform (Intelligent Tutoring Supported Collaborative Learning) allows individual and group learning [2].In individual learning, a single learner engages with ITSCL, which employs the same teaching methodology used in traditional ITS.Furthermore, through comments and instructions, ITSCL enables students to reflect on responses from one another and share the knowledge they have learned.Due to this knowledge-sharing and rating approach, students can reflect on their understanding, analyse their responses, and revise or update their answers, promoting high levels of collaboration and shared knowledge.Following the conclusion of each student's response, ITSCL evaluates each response and chooses a more genuine/matched answer as the collaborative response.This learning process aids students' individual and collective responsibility in a collaborative group.ITSCL collaborative learning interface is presented in 1.

Dynamic Group Formation (DGF)
The Dynamic Group Formation (DGF) technique using Intelligent Tutoring Collaborative System (ITSCL) is a web-based collaborative learning environment for computer programming modules that allows learners to collaborate and chat with each other during learning sessions [6].DGF achieved heterogenous balanced groups based on learners' knowledge level that has been assessed through short quizzes.Students are shuffled into different groups based on their knowledge levels after each activity has been permuted.This approach suggests that balanced, diverse groups can be created during learning.The DGF methodology is based on learners' knowledge level and learning style, where learners' initial clusters are formed and then swapped after each permutation until heterogenous balanced groups are formed.The DGF methodology is illustrated below in Figure 2.

EVALUATION 3.1 The method
For user experience evaluation, we have used a hybrid approach consisting of a user experience survey using a questionnaire and aspect-based sentiment analysis (ASBA) of learners' comments at the end of the survey.A questionnaire is the most common evaluation method for user experience [7], while aspect-based sentiment analysis is an effective technique to extract users' sentiments regarding each aspect of experience [8].For the survey, fourteen closedended questions on a Likert scale with five options ranging from "strongly agree" to "strongly disagree" were used in the questionnaire.After using the DGF-ITSCL, the learners were requested to fill out the questionnaire.They are also requested to comment regarding their opinion, which can consist of strengths/ weaknesses and other user-experience parameters of the DGF-ITSCL.The learners' comments are then used to apply ASBA to extract user-experience aspects and their emotions behind each aspect.

Participants and data collection
For the user-experience survey, we have targeted undergraduate students of Computer Sciences.The online survey questionnaire consists of a system prototype link to use the system and then answer the close-ended questions in the questionnaire along with comments and opinion options.Through the questionnaire, we received 57 responses from undergraduate students (63% male and 37% female).The questions asked at the end of using the DGF-ITSCL prototype are given below in Table 1.

Aspect-based sentiment analysis (ASBA)
Aspect-based sentiment analysis (ABSA) is a natural language processing (NLP) technique that focuses on recognising and analysing the sentiment of particular aspects or qualities of a good, service, or entity [9].ASBA is an effective technique to evaluate a system's user experience by analysing users' opinions and sentiments about different aspects of the system.It can provide a more detailed and Figure 2: Dynamic Group Formation Methodology [6] nuanced understanding of users' experiences than traditional sentiment analysis methods.For instance, ABSA can assist in determining which features or functionalities of a system customers value the most or find the most annoying.It can also determine users' satisfaction with the system's overall user interface and experience [10].
The authors used ASBA to extract learners' sentiments about different aspects of DGF-ITSCL by utilising the comments/opinions collected through an online questionnaire.For ASBA, the authors implemented Valence Aware Dictionary and sEntiment Reasoner (VADER) lexicon-based tool.VADER is very helpful for ABSA since it can accurately capture sentiment polarity and intensity for specific features or attributes of a text by considering the valence of individual words and their context and syntax.VADER has proven effective in numerous ABSA applications, such as sentiment analysis of social media posts and consumer feedback analysis [11].

RESULTS AND DISCUSSION 4.1 Questionnaire Results
Table 2 shows the participants' responses and experience of using DGF-ITSCL.

Aspect Sentiment Analysis (ASBA) Results
The responses collected through open-ended questions 13 and 14 were analysed.The aspect-based sentiment analysis technique extracts users' comments and opinions regarding DGF-ITSCL userexperience aspects.For ASBA, five user-experience aspects, i.e., user-friendly, usefulness, accessibility, value, and effectiveness, were pre-defined to detect and classify users' emotions for each aspect into positive and negative emotions.The ASBA results are as follows:  77.9% were positive, 10.4% were negative, and 11.7% were neutral comments, as depicted in Figure 3.

4.2.2
Sentiment classification of user comments into pre-defined aspects.ASBA was applied to extract positive and negative sentiments about each aspect of DGF-ITSCL by using the VADER technique.The number of positive and negative sentiments of each aspect is illustrated in Figure 4.

CONCLUSION
In this paper, the authors have proposed a hybrid methodology to evaluate the user experience of Dynamic Group Formation (DGF) in Intelligent Tutor Collaborative Learning (ITSCL).Dynamic Group Formation is considered an essential aspect of collaborative learning, where learners can work together in the same group to improve their learning abilities.ITSCL provides a novel approach of dynamic group formation where students with the same learning style are combined in the same group and then swapped with another group based on their knowledge and academic performance to achieve heterogenous balanced groups.Initially, a survey was conducted through an online userexperience questionnaire for user-experience evaluation.The learners were asked to interact with the DGF-ITSCL system and then complete the questionnaire with 14 user-experience questions.Moreover, aspect-based sentiment analysis using the VADER technique has been used to classify learners' comments into positive, negative, and neutral classes and then extract sentiments about pre-defined user-experience aspects of DGF-ITSCL.The results showed that the presented DGF-ITSCL is a useable, helpful, and effective platform that can be utilised to form heterogenous balanced groups of learners during online collaborative learning.In such a way, this evaluation completes the previous evaluation of the system in terms of learners learning gain and knowledge level improvement through DGF [6].Our future work involves the deployment of DGF-ITSCL at a large scale in academia and providing the same system across different platforms, i.e., mobile application and offline desktop system.

4. 2 . 1
Classification of comments into positive, negative, and neutral classes.The user comments were initially classified into positive, negative, and neutral classes using the VADER technique of NLP.This classification aims to depict the overall responses to understand users' satisfaction.A total of 165 responses were collected, of which

Figure 3 :Figure 4 :
Figure 3: Comments classification into positive, negative, and neutral classes

Table 1 :
Questionnaire questions