Enhancing User Experience: Virtual Assistants in Greek University Helpdesk Service

It is commonly accepted that we live in a digital era of constant changes. Digital revolution has brought significant changes in the way we live, work, and interact with each other, with messaging apps turning to the centerpiece of our communication. Individuals and especially youngsters are constantly connected through devices for both social and entertainment reasons. This growing enthusiasm for messaging apps brought the use of bots as supplementary virtual assistants into the spotlight. Examples of companies employing bots into their business chats examples marked that chatbots contribute to a better user experience while at the same time reduce personnel workload. The results raised the question whether the introduction of virtual assistants will benefit the everyday life of Greek Universities. To examine our hypothesis in the realms of University of Western Macedonia, we introduced the “bot-UoWM”, a zero- code conversational chatbot that supports Greek Natural Language, in well-known social media platforms. To evaluate the system, we conducted an empirical study using a questionnaire on 80 participants. The obtained survey data were analyzed using a combination of quantitative and qualitative techniques. The evaluation results show that our system improves user acceptance and understanding of the results. They also indicate that bots could help different users with their seeking tasks. Our results and discussions highlight the impact of using visual and interactive features as assistants for higher education.


INTRODUCTION
Students nowadays prefer texting through digital devices over reallife interactions.They would rather ask questions and get rapid responses than spend time looking for information.This desire for shorter search times allows individuals to devote more time to evaluate the solutions presented.However, it is critical to Figure out if this desire is shared by older people.This research additionally examines how the academic community might profit from the changing communication landscape.The objective of this study is to investigate the effectiveness of utilizing chatbots to reduce administrative workload at universities while boosting student involvement.To achieve these goals, we developed "bot-UoWM, " a zero-code conversation chatbot.This chatbot recognizes and answers in Greek Natural Language and has its own Knowledge Database.
We distributed bot-UoWM throughout several social media platforms, capitalizing on the rising popularity of messaging applications.This technique enables us to reach a big and diverse audience both inside and outside of the University of Western Macedonia (UoWM).Participants in our study connected with bot-UoWM via known and comfortable paths, which simplified and made the assessment process more accessible and meaningful to them.
The rest of the paper is organized as follows: Section 2 summarizes the key contributions of previous research related to our topic of interest.Section 3o outlines the characteristics of the bot-UoWM, the implementation of the system and the fields of use.In Section 4 we introduce the evaluation process of bot-UoWM and present the methodologies of data collection and analysis.Section 5 presents the results, organized by research question while in Section 6 we provide a discussion on the meaning and significance of the findings, we also address any potential limitations or threats to the validity of our study.Finally, in Section 7, we conclude the paper.

RELATED WORK
With the rise of GPT3 and the introduction of ChatGPT more and more organizations are turning their attention to the use of chatbots as virtual assistants or even as supplementary assistants for their organization units.The use of chatbots for everyday life tasks, turned to a trend in the past 3 years and especially after COVID-19, even though it is not a state-of-the-art concept.The first use of AI as an assistant was more than half a century ago.Prominent examples include ELIZA; an early Natural Language Processing program that simulated the communication between humans and machines [4] developed by Weizenbaum in [4] , A.L.I.C.E.; Artificial Linguistic Internet Computer Entity the first online chatbot with a discussion ability was introduced by Wallace in (1995) and Smarter-Child.[15], was available on Messengers like America Online (AOL) and Microsoft (MSN) in 2001 and it marks the bots' first appearance in social platforms.In 2017 Georgia University introduced Pounce, the first virtual assistant that helps and guides freshmen students with enrollment tasks.Following the example of GU, Universities around the globe slowly started introducing different versions of virtual assistants (i.e., Botter, CourseQ) to help students with academic tasks, reduce the staff's workload and make the everyday life in University more pleasant.To the best of our knowledge, no present chatbot provides access to information on specific university's operations using the Greek Language.This study aims at the exploration and coverage of the aforementioned gap.We went into the design process knowing that our chatbot would not be able to answer every question of the user, but we nevertheless set out to provide an interesting and positive user experience.

BOT-UOWM DESIGN GOALS AND TECHNICAL IMPLEMENTATION 3.1 Design Goals
Before starting the implementation of the chatbot itself, a process of listing technical requirements and non-technical requirements was undertaken.The requirements that were defined include the following aspects: Availability of the system: The chatbot should be always available to users.
Language assistance, Greek language assistance is a crucial prerequisite for efficiently catering to the university community.
Natural Language Processing (NLP).The chatbot should be able to interpret and respond to user inputs that involve grammatical or typographical errors.
Constant Accuracy Training to guarantee accuracy and upto-date information.
Quick responses for a smooth and effective conversation flow.Short and Readable Responses, allowing users to rapidly absorb the offered information.
Scalability on handling multiple requests at once Device Flexibility to be accessible and usable on desktop computers, tablets, and smartphones.
Compatibility with several messaging platforms Friendly and approachable persona to create a positive user experience and encourage engagement.

Tool and Platform Used
Our research purpose is to examine the need for virtual assistants in universities.We install ready-to-use technologies instead of focusing on system development from scratch.Several factors influenced this decision, including the authors' lack of expertise building such systems, which highlighted the importance of detailed documentation, accessible helpdesk support, and an active user community.This method democratizes bot creation, making it accessible to those with little to no programming skills.We used Microsoft Azure technologies for the scope of our investigation.Microsoft Bot Framework can cover all targeted platforms as FB Messenger, Skype, and Teams.MS Azure platform can serve both the Bot and the Bot backend services.From the security reasons Azure is great option since Skype and Teams are also Microsoft products which are natively connected with Azure.Another significant advantage of adopting Microsoft Azure was its support for Greek (NLP) and the ability to combine different sources as a custom database.
The list of tools is: • Language Studio for NLP database creation and integration.
• Bot Framework Composer to connect to Azure and publish the bot.• Bot Framework Emulator for bug identification if any exist.
• Azure provides connection with the communication channels.Using the aforementioned platforms, the whole process of creating, training, testing and implementing the bot-UoWM took 2 weeks.The second author who was responsible for the bot-UoWM built process had no previous knowledge or expertise on the subject.

System Architecture
Bot-UoWM uses the Teams and Skype platforms for communication.As interaction inputs, the bot takes text, touch, and interactive cards, adjusting to the user's device (e.g., desktop, tablet, smartphone).Additionally, to address a range of user demands, we merged location provider, point of interest provider, file provider, and link provider capabilities.To ensure user experience, we established an "unknown intent trigger" that activates when users make requests that are unrelated to university operations or include objectionable information.If the answer to a given question is not included in the database or requires additional information, the bot encourages the user to contact the administrative office.Figure 1

presents the architecture of bot-UoWM. User Interface and Integration in Messaging Apps
The result of the implementation process is presented in this Section (Fig 2-5), including some examples of the most relevant functionalities.In Figure 2, the user specifies his role, and the bot responds with all available categories relevant to his role, retrieved from its knowledge base.The functionality of active links and the location provided are shown in Figure 3. Figure 4 showcases a user's request for a file, with the bot offering the option to download it.Figure 5, demonstrates the scenario where the database lacks information, prompting the user to contact the administrative office for assistance.
The first two Figures represent screen shots of the Skype implementation, while the other two display the Teams' user interface.

EVALUATION
To evaluate the bot-UoWM, we performed a controlled experiment.The primary goal was to assess the system's performance in recognizing participants' demands.In addition, the system's usability was assessed using a customized online survey.

Participants
We conducted a two-group evaluation study to explore the effectiveness and quality of bot-user interaction.
• The first group of individuals had to perform a set of tasks related to retrieving information from the public sources (i.e.website, internal regulation) of the university.This group of users did not interact at all with bot-UoWM.• The second group of individuals had to perform the same set of tasks related to information retrieval, as Group1, with the help of bot-UoWM.
Our target is to analyze how the bot influenced the responses and the interactions within the two groups.
The bot-UoWM was evaluated by a total of 80 people.The people that participated in the experiment presented different backgrounds: The 80 people were then split equally into two groups A and B. We included an equal amount of people with different backgrounds in both the groups.Group A, 40 people, received only the set tasks (questions that needed answers) while in Group B the 40 participants, received a questionnaire and the appropriate links to access the bot-UoWM.

Questionnaire
The Questionnaire that includes the set of tasks each user has to perform consists of three parts.The first and second parts are the same for both groups, while in the third different questions were formed for those using and not using the bot-UoWM.
First part: the first part of the questionnaire consists of 7 general questions, that help us understand the familiarization of the participants with the use of bots and whether they have a positive or negative point of view regarding interacting with chatbots.
Second part: the second part of the questionnaire consists of 12 Typical and Non-Typical Tasks related to retrieving information for every-day activities within the University.After completing each task, the participants were asked to provide their answers and the time required to find the answers.
Third part: the third part of the questionnaire consisted of 4 User Satisfaction Questions.Depending on the target group, participants were asked to a) provide feedback and suggestions on the current available processes of UoWM, b) rate the bot-UoWM and provide improvement suggestions.
To minimize the potential biases in participants' answers and to ensure consistency in our data collection we employed a combination of 5-scale Likert Scale answers (General Questions, Tasks) and open-ended questions (User Satisfaction Questions).

Research Questions
In this section, we present the research questions we answered with the help of the presented experiment: [RQ1] Does the availability of bot-UoWM reduce the effort needed to complete tasks?
The goal of this research question is to determine whether the existence of bot-UoWM,causes users to exert less effort when completing various tasks and answering questions regarding the university's services and information.
[RQ1] Does the availability of bot-UoWM reduce the effort needed to complete tasks?This question seeks to analyze the findings and identify specific types of tasks or queries that demonstrate the most significant benefits from using bot-UoWM, providing insights into the bot's areas of effectiveness and utility.
[RQ3] Does the availability of bot-UoWM increase the engagement of users?
This research question investigates whether the presence and interaction with bot-UoWM contribute to higher engagement among users.Each of the three parts of the given Questionnaire provide insights for our research questions.

RESULTS
[RQ1] Does the availability of bot-UoWM reduce the effort needed to complete tasks?
The findings from the given Questionnaire suggest that participants on Group B (bot-UoWM Users) require less time to complete tasks.They rated bot-UoWM's responses as correct and accurate.The process of collecting findings within each category using the bot took around 36 seconds on average, whereas users without access needed more 4.1 minutes on average to answer questions related to technical matters (Table 1).Participants of Group described information retrieval process more challenging than expected, with 94% of them considering their answers to be incorrect.Only 5 participants of Group A managed to complete all given tasks.
[RQ2] Based on the results, which types of tasks can we identify that benefit most from the use of bot-UoWM?Based on the results provided from both groups, we can identify specific types of tasks that would benefit the most from the use of bot-UoWM.Notably, tasks characterized by lengthy information retrieval related to technical issues or associated with complex processes of UoWM (and topics that are not readily available on the university's general website and require in-depth navigation) were challenging for participants in Group A (Figure 6) who did not have access to the bot-UoWM.In contrast, questions concerning student-related information the need of a virtual assistant seems somehow trivial (Figure 7).
[RQ3] Does the availability of bot-UoWM increase the engagement of users?
While user engagement may not be directly measurable in quantitative terms, we assessed it through user satisfaction feedback in the Questionnaire.This qualitative data provided useful insights into how users perceived their interaction with the bot (Table 2).Moreover, in the open-ended questions users left the following comments: "I think it would be useful if the chatbot was introduced in more faculty structures", "The chatbot gave correct answers.It can help the students of the faculty", "Fast responses and user-friendly.Overall, very good, quick, and effective.Perhaps, an innovative addition would be the ability to accept voice commands." An important hint of increased user involvement was the number of queries handled by the bot.Even though Group B participants were given a total of 12 tasks to complete using the bot analytics showed that the total amount of hits was over 5000, causing a twoday failure for security reasons, when the expected and needed amount of hits to complete all tasks for all participants was roughly over 500.Another indicator of user engagement is the time of each session with the bot.Participants in their answers mentioned that the process of answering question took them less than 12 min to complete, but each session with the bot lasted around 15-10 minutes, it became clear that they spent more time talking with the bot than expected, indicating an enhanced level of engagement.Data derived from Azure's analytics and telematics services confirms that end users did ask more inquiries than expected.Taken together, these findings strongly suggest that the bot's availability at UoWM will considerably increase user engagement, as users actively interact with the bot, seek information, and offer feedback on their experiences.

DISCUSSION
In this section, we first interpret the results of the research questions and then we discuss the threats to validity of the research.

Interpretation of Results
The findings of our RQs suggest that not only does the bot reduces time and effort needed to browse and retrieve info but also makes the process more pleasant for the user.The 24/7 availability of services has a big impact on user satisfaction.Implementing botUoWM on handling common requests and more technical user requests would lighten the workload of the secretariat.Based on the results in.Chatbots have potential to replace at least of 50% of the Support or Helpdesk service reducing workload and creating a seamless smooth experience fir the user.A well-trained bot powered by AI and Cognitive services has the potential to replace Support or Helpdesk services in the academic community by providing correct and quick information.
When comparing university bots, bot-UoWM differs from other university bots in that it employs a "No Code" implementation technique, making it a flexible and user-friendly solution.The fact that bot-UoWM can function without the requirement for programming skills makes the implementation process easier for individuals without coding experience.Additionally, by supporting Greek, it appeals to a wider linguistic audience and encourages inclusivity.More customization options are available with Pounce and Botter, which were developed using Python and JavaScript but require programming experience.Additionally, Pounce and Botter are largely aimed at students, but bot-UoWM invites users from all backgrounds.bot-UoWM's adaptability and user-friendliness, coupled with its accessibility via Skype and Microsoft Teams, position it as a compelling choice for institutions seeking a chatbot solution that bridges language barriers and serves diverse user groups.

Threats to validity
This section presents the validity threats in our experiment.We maintain the common distinction between internal validity, which refers to the cause-effect inferences made during the analysis, and external validity, which concerns the generalizability of the results to different contexts.[14] Subjects.To reduce internal validity, we contacted an a priori selection of subjects and chose only participants that know how to use at least one of the social media platforms where the bot-UoWM was implemented.Moreover, when grouping the subjects, we made sure that their familiarity with the institution was evenly distributed across the groups.Lastly none of the subjects were aware of the actual research questions (although they may have guessed).
Questions.The questionnaire was designed by the authors of this paper, and therefore may have been biased towards bot-UoWM (as this tool was also designed by several of the authors).To avoid this threat, we used special templates and utilized SUS techniques.[1] Regarding generalizability of study outcomes is referred to as external validity, we identified two possible threats.Our research focuses on UoWM, and therefore it is not clear whether the results can be generalized to all higher education domains or settings.Additionally, the sample size was relatively small and, thus, the findings of the study should be confirmed by a larger sample size and extended to other populations of different age groups or backgrounds.As a result, we encourage future studies to replicate our findings using bigger sample sizes and their custom Knowledge Database.As our research on this topic is ongoing, we plan to address the aforementioned threats by expanding our sample size and re-exploring our hypothesis.

Future studies should focus on
Open approach, our next step involves exploring a no-code methodology that leverages open-source software.
Generalization and effectiveness of bot-UoWM, by conducting tests in different university institutions.This will allow researchers to understand how well the bot performs across various academic settings and identify any specific challenges or adaptations needed for different environments.
Expansion of knowledge databases beyond academic matters, incorporating information about life outside University (i.e bus routes, important sights) can greatly benefit students, especially freshmen.
Engagement options, making bot-UoWM more inclusive and accessible, incorporating voice or speech-based interaction options can ensure that users, regardless of their abilities, can benefit from the bot's assistance.
Implementing other mediums, integrating bot-UoWM with the institution's website and other more traditional platforms, will provide users with alternative entry points.

CONCLUSION
Our research highlights the effectiveness of bot-UoWM to enhance user interactions, especially when dealing with complex or difficultto-find information.The results provide a positive outlook for the introduction of virtual assistant chatbots in organizational and academic settings, functioning as valuable resources for users looking for prompt and trustworthy assistance.Users were very satisfied with the use of the chatbot, mainly because it provided them with instantaneous feedback at different times, without encountering any delays in the interaction process.By introducing bot-UoWM through popular social media channels, we were able to capitalize on the increased interest in messaging apps and reach a more diversified as well as young audience.This strategy perfectly aligns with our overarching goals of improving administrative effectiveness and increasing student involvement at the University of Western Macedonia (UoWM), as it not only minimizes the administrative burden on the university's secretariat but also increases engagement among students and other stakeholders.
and members of the Academic community of UoWM.(40 people) • People with previous interaction with the UoWM, but not members of the Academic community of UoWM.(20 people) • People that are unfamiliar with UoWM processes.(20 people)

Table 1 :
Time Average