Text Generation with Personalization Features for Information Presentation in Decision Support

There is a high demand for supportive technologies for decision-making processes. An effective way to support user decision making is to present necessary information in a text format that is easy for users to understand. In cases where different users have different decision-making objectives and preferences, personalization of information presentation is effective in improving user utility by presenting information that is appropriate for each user, rather than presenting the same information to all users. In this study, we developed a text generator with personalization features of information presentation for the decision-making situation of purchasing betting tickets for bicycle racing, a publicly operated sport in Japan. We developed a rule-based text generator that assumes multiple typical user groups in a decision-making scenario and generates appropriate text for each user group. The developed text generator takes as input information such as race results predictions based on deep learning models and personal information about the racer, and outputs texts that explain this information. We conducted an evaluation experiment with human experts and found that the generated text is valuable information that improves user’s utility as text for each user group.


INTRODUCTION
Decision-making is the process of selecting a single action with the highest utility among multiple actions; there is a high social demand for assistive technologies for this process.When the utility function of a decision-making user is known, the action that maximizes the user's utility can be automatically selected through machine learning or mathematical optimization.However, when the user's utility function is unknown, the decision support that presents information to maximize the user's utility is implemented, assuming that the final choice of the action is made by the user.
In particular, when the amount of information involved in the decision-making is large and complex, it is essential to support the decision-making process by presenting the information to the user in an easy-to-understand format.Various formats can be used to present the information, such as text, tables, diagrams, and charts.Among these, the text format is typically preferred and employed in various scenarios.For example, in medical research, it has been reported that more accurate decisions are made when physiological data on a patient's condition are presented in a textual format rather than a graphical representation [7].
The user's utility function depends on the user's objectives and preferences, and the effective information for the decision support varies from one user to another.Therefore, to improve the user utility, the information being presented must be personalized for each user, rather than presenting the same information to all the users.
Consequently, we developed a text generator with personalization features for decision support.In this study, we focus on the problem of purchasing betting tickets for bicycle racing, a publicly operated sport, and generate text presenting the predictions of the race results along with personal information on the racers to support the decision-making process.We equipped the text generator with personalization features by extending the template-and rulebased method to generate explanatory text for bicycle racing, as reported in our previous study [17].Furthermore, we conducted an evaluation experiment with human experts, which demonstrated that the generated text provides valuable information that maximizes the utility for each user group.
The contributions of this paper are twofold.
• We developed a text generator that explains personal information of racers using publicly available data.We have demonstrated that the generated text provides valuable information for viewing-oriented users, through evaluation by human experts.• We conducted a human expert evaluation experiment to compare an existing text generator for predicting-oriented users and the developed text for viewing-oriented users.The results presented that the information of value to each user group was different, indicating the effectiveness of using different text generators for different types of users in personalization of information presentation.
While it is important to present valuable information to users, the personalization of information presentation has not been actively developed in the field of betting ticket assistance thus far.It is hoped that the results of this paper will stimulate research and development on the personalization of information presentation.

RELATED WORKS 2.1 Data-to-Text
Data-to-Text is a task that generates descriptive text from table data and other structured data.Data-to-Text datasets include the E2E NLG Challenge [10], a dataset for generating restaurant descriptions, and RotoWire [16], a dataset for generating summaries of basketball game results.Further studies include generating weather comments from weather simulation results [8] and generating market comments from stock price data [9].Text generation in bicycle racing is the task of generating text from table data such as racer information, which corresponds to Data-to-Text.The text generation process employed in this study differs from generic data-to-text since it involves race results, which cannot be explicitly provided at the time of generation.

Text Generation Methods
There are two methods of automatic text generation: neural language models and rule-based methods.
Recent neural language models typically use the Transformer [15], in which a model is trained by maximizing the log-likelihood using a data set consisting of pairs of input data and the corresponding correct text.By finetuning pre-trained models like GPT-2 [12] and T5 [13], It is possible to generate fluent and high-quality text in Data-to-Text tasks [4,6].However, neural language models face the drawback of sometimes generating sentences that differ from the facts in data-to-text.A similar problem has been observed in the text generation in bicycle racing [17].In recent years, large-scale language models(LLM), starting with GPT-3 [1] and including PaLM [3] and , can generate fluent, high-quality text without additional training.
The rule-based method uses manually set templates and requires work to design the templates and if-then rules.However, it has the advantage of being able to guarantee the grammatical and factual accuracy of the generated text.In this study, we selected a rulebased approach because of the accuracy and ease of controlling the generated text.We extended the template of a bicycle race description generator from our previous study [17] conducted on implementing personalization features.

Personalized Text Generation
Personalization of text generation, in which the generated text changes according to the user, has been studied in various domains.A system called PASS was [14] developed to generate different soccer match reports for fans of the winning and losing teams.Additionally, a study was conducted to apply the PASS system to generate explanatory text on the quality of life (QOL) of cancer patients after their treatment [5].This previous study aimed to assist patients in making treatment decisions by generating different explanatory statements based on the different magnitudes of the impact of the treatment on post-treatment quality of life.These studies used a template-based sentence generation method.One example of research on the personalization of text generation using neural language models is the generation of product descriptions for ecommerce sites [2].This study differs from previous studies in that it focuses on the decision-making problem of purchasing betting tickets in bicycle racing, where user utility has both extrinsic and intrinsic utility components.

MODELING OF DECISION-MAKING
This section describes the model of user decision-making process assumed in this paper.

Maximize Utility by Presenting Information
Figure 1 presents an overview of decision-making support by presenting information.In this study, we assume that the user makes decisions based on the utility function:  :  ×  → R. Here,  ∈  represents an action, and  ∈  represents information.Given a specific piece of information, , the user selects an action, argmax  [ (, )], which maximizes the utility,  (, ).The utility associated with an action depends on the information provided to the user.Consequently, the same action may yield different utilities, based on the type of information available to the user.The goal of decision support is to present information, , that maximizes the user utility, specifically, max In real-world decision-making problems, there are cases in which the utility function differs from user to user due to differences in user objectives and preferences.In this case, to maximize the user utility, the information being presented must be personalized for each user, rather than presenting the same information to all the users.
Utility in real-world decision making can be divided into two categories: extrinsic utility and intrinsic utility.An example of a decision for which extrinsic utility is the primary utility is stock investment.Examples of decisions in which intrinsic utility is the primary utility include viewing the sport or watching a movie.Extrinsic utility tends to be objectively determined, while intrinsic utility tends to be subjectively determined.Decisions in which extrinsic utility is the primary utility require less personalization of information presentation than decisions in which intrinsic utility is the primary utility.Also, decisions in which intrinsic utility is the primary utility are more difficult to personalize the presentation of information than decisions in which extrinsic utility is the primary utility.Therefore, a decision-making scenario with both extrinsic and intrinsic utility is desirable in the study of personalization of information presentation.This study deals with the decision to purchase a betting ticket for publicly operated sports, which has elements of both stock investment, an example in which extrinsic utility is the primary utility, and viewing the sport, an example in which intrinsic utility is the primary utility.

Sport.
A typical race comprises a maximum of nine racers who compete for the fastest time to complete a lap around a racetrack, called a bank, over a distance of 1,500 m-3,000 m.Bicycle racing is a relatively popular professional sporting event in Japan, with about 50 races held daily.The Olympic event known as Keirin was inspired by Japanese bicycle racing.In Japan, bicycle racing racers are often chosen to compete in Keirin and other world-class cycling events.
In Japanese bicycle racing, there is a team-like element called the "line, " which is not observed in other track races.During a race, the competitors form vertical lines and cooperate with other racers that belong to the same line.Racers along the same line cooperate with each other until the last straight line before the finish line, where they compete with the other racers in the same line, with each individual aiming for the first place.Racers from the same region often form a line together and the information on the racers that form the line is made public before the race begins.

Sports Betting.
Bicycle racing in Japan is a publicly operated sport for betting.Spectators buy betting tickets that predict the race results and receive a payout if their predictions are accurate.The tickets are sold based on the pari-mutuel system; the odds and ratio of the payout are determined by the percentage of votes cast by all the spectators.There are two types of bicycle racing betting systems: exacta, in which the first and second placed racers are predicted to finish in the same order, and trifecta, in which the first three placed racers are predicted to finish in the same order.The convention for writing the predictions in the text format is to write "1-2" when the first place represents racer number 1 and the second place represents racer number 2.

TEXT GENERATION METHOD
In this study, we developed a text generator with personalized features by extending the text generator developed in our previous study [17].Figure 2 presents an overview of the text generation process.The text generator generates text by using a rule-based method that manually sets the templates.The template contains slots, and the text is generated by assigning the corresponding values, such as the racer's name and past performance, to these slots.

Data Used for Text Generation
The data used for text generation can be divided into two main categories: public data on bicycle racing, and data obtained through the conversion of public data.
The public data includes the racers' past performances, personal information, odds ratios, and line information.The racers' past results includes the percentage of first-place finishes in the last four months, percentage of solidarity, percentage of last place finishes in the last four months, and total number of wins.Personal information on the racers, such as birthdays, favorite foods, and past sports experiences, was also included.Since our previous study [17] did not deal with racer's personal information such as favorite foods, a new database was constructed by scraping from official websites that publish racer's personal information.Personal information about the racers was collected, including physical characteristics such as height and weight, friendships such as training buddies and mentors, favorite foods, grades during training school, and sports experiences during school.
The output of the race result predictor [17] from the machine learning model was used as the transformed data obtained from the public data.For the outputs of the predictor, we used the probability of a racer being in first place at the time of the last half-lap, the probability of a racer being in first place, the probability of the occurrence of an exacta, and the probability of a racer being in first place conditioned on the racer being in first place at the time of the last half-lap .
The predictor was trained to take a sequence of competing racers,  = [ 1 ,  2 , ...,  9 ], as the input and predict the sequence of race results  = [ 0 ,  1 ,  2 ,  3 ].Here,   represents the feature vector of the racer with bib number, ,  0 indicates the bib number of the pacer, and  1 ,  2 ,  3 denotes the number of bib racers finishing in the first, second, and third places, respectively.The transformer encoder-decoder model was employed, which includes various features such as the track information, past performance of the racers, and the relationships between the racing lines.The model was trained on approximately 100,000 races spanning five years, from 2013 to 2018, and was evaluated on approximately 20,000 races from 2019.The results demonstrate that the model achieves a prediction accuracy that is comparable to or better than that of the betting odds.Furthermore, considering the expected returns from the betting tickets, a return on investment exceeding 100% was achieved [17].
These data were used to fit the values to the slots in the template and determine whether the template could be used.

Template Used for Text Generation
To generate that text that can improve the utility of both the prediction and viewing-oriented users, we set up templates that we considered appropriate for each user.

Templates for
Predicting-Oriented Users.The text for predictingoriented users is the same as that developed in our previous study [17], and this text and predictions are in production for the Japanese language as a web service named AI bicycle racing 1 .
For predicting-oriented users, we set up templates that describe predictive information about race results, racer past performance information, and line information.Typically, the order of arrival in public games can be categorized into several types: first favorite to win, second favorite to win, dark horse, possible second place, and long shots.In this paper, the ticket with the highest probability of occurrence is selected as the first favorite to win, while the one with the second highest is the second favorite to win, the ticket with high expected payout value is the dark horse,and a text with multiple predictions is generated.
The basic structure of the configured template and an example template are presented below.

Description of the ticket with the highest probability
The first favorite to win is {{ticket1}} with a hit probability of {{probability1}}%.{{racer1}}, who has the highest quinella rate among the contestants with {{quinella rate}}% in the last four months; will come out at the top in the last half lap.Subsequently, {{racer2}} will overtake him from behind {{racer1}} to take the first place.

Description of the ticket with the second highest probability
The second favorite to win is {{ Ticket 2 }} with a hit probability of {{ probability2 }}.Be wary of {{Racer name3}}.

Description of the ticket with high expected payout
The dark horse is {{ticket3}}.With a hit probability of {{prob-ability3}}% and odds of {{odds}}, the expected payout is high.

Templates for
Viewing-Oriented Users.For viewing-oriented users, we set up templates that describe the racer's personal information that would trigger their attention to the racer when they view the race.We set up templates to describe a racer who is nearing a milestone in terms of total wins or total appearances, or whose birthday is close to that of the racer.We used personal information about the racers: their favorite and least favorite foods, their performance during their training school years, and their sports experiences during their school years.The basic structure of the configured template and an example template are presented below.

Description of racer's birthday, number of races participated or number of wins
If {{ racer}} wins today, he will have won a total of {{ wins }}.

Description of the results of the previous race and the predicted win probability for this race
Although he finished {{ latest result }} in his last race, he has a chance for revenge since he has the highest predicted probability of winning {{ win probability}}% in today's race.

Description of personal information about the racer
In middle school, he played {{ sports1 }} and in high school, he played {{ sports2 }}.His favorite food is {{ food1 }} and least favorite food is {{ food2 }}.

EXPERIMENT
This section presents an experimental analysis in evaluating the generated text by a human expert.

Experimental Setup
The purpose of the experiment is to evaluate whether the developed text generator with personalization features can maximize utility for each group of users in a given decision-making scenario.However, measuring user utility is difficult.Therefore, as the first step in this study, we evaluated the information by having the experts answer the question, "Is the information presented valuable to the user?"Ten samples of each of the two types of generated text were evaluated by five domain experts in bicycle racing.All evaluators were employees of a company that provides a service of selling bicycle racing betting tickets and are private bicycle racing users.Evaluators scored each of the two types of samples on a five-point Likert scale (5 = strongly agree, 4 = agree, 3 = somewhat agree, 2 = somewhat disagree, and 1 = disagree).We did not use the criterion "neutral" so that evaluators would express their opinion more clearly.In this framework, scores of 3 or higher are interpreted as valuable.The questions were "Is the information valuable to predicting-oriented users?" and "Is the information valuable to viewing-oriented users?"The evaluators were asked to evaluate only the generated text.Since the evaluators were presented with only the generated text, their evaluation did not consider whether the predictions were accurate or not.The text used in the experiment was Japanese, and all the evaluators were native Japanese speakers.
We conducted Brunner-Manzel tests to determine if there was a difference in scores for each generated text.The significance level was set at 5% for all the tests.A significance level of 2.5%, corrected using the Bonferroni correction, was used to perform the tests for each of the two questions.
After the experiment, the evaluators were asked to provide feedback on the information that they considered appropriate and valuable for each user group.

Results and Discussions
Figure 3 presents the frequency distribution of the overall evaluation scores for each question, and Table 1 presents the mean and median values of the evaluation scores.The results of the Brunner-Manzel test indicate significant differences between text for predicting-oriented users and text for viewing-oriented users for both questions "Is the information valuable to predicting-oriented users?"(with a p-value of 5.4 − 17 < 0.025) and "Is the information valuable to viewing-oriented users?"(with a p-value of 0.023 < 0.025).Text for predicting-oriented users received a mean score of 3.64 for question "Is the information valuable to predictingoriented users?" and text for the viewing-oriented users received a mean score of 3.38 for question "Is the information valuable to viewing-oriented users?",It is evident that both scores are better than 3 (where 3 signifies 'somewhat agree').These results indicate the effectiveness of the developed text generator in personalization features.
Table 2 presents examples of text for predicting-oriented users with the highest and lowest mean scores for question "Is the information valuable to predicting-oriented users".Table 3 presents examples of text for viewing-oriented users with the highest and lowest mean scores for question "Is the information valuable to viewingoriented users".More than half of the scores("Is the information valuable to predicting-oriented users") for text for predicting-oriented users were 4 or 5. Conversely, more than half of the scores("Is the information valuable to viewing-oriented users") for for text for viewing-oriented users were 2 or 3.In the feedback from the evaluators after the experiment, two evaluators responded that the information on the racers who had recently had a birthday and the information on the racers who were close to the number of milestone appearances were appropriate as valuable information, respectively.This indicates that the content of the template for viewing-oriented texts was not considered appropriate by some evaluators, which may be one of the reasons for the relatively low evaluation rating of the viewing-oriented text.

CONCLUSIONS
In this study, we have developed a text generator with personalization features to support users in purchasing betting tickets for bicycle racing.The generated text presents information that is important to a user's decision-making process, such as the racers, lines, and predicted race results.We implemented personalization features that assumes the typical groups of users based on the utilities they value and generates text for each group with different information on the racers and tickets.The results of an evaluation experiment with human experts demonstrated the effectiveness of the system for personalized text generation for each user group.
Two future prospects are the estimation of a user's utility function and the generation of texts for more diverse users.
The text generator with the personalization features developed in this study can output text that is suitable for a user if the user's utility, or user group, is known; however, in situations where the user group is unknown, a function to estimate the user's preferences is required.It is necessary to develop a method to estimate the utility by using historical information on the user's past decisions, or a method to estimate the utility while collecting information regarding the user, starting from a situation where there is no historical information at all.
In this study, personalized text was generated for two groups of users: those who are focused on predictions and those who are focused on watching the game.To accommodate more diverse users in the future, it will be essential to consider text generation methods that utilize LLM, such as GPT-4, because it is not realistic to use only rule-based methods in which templates are set manually.Predicting-oriented users suffer the disadvantage of a missed prediction if the given prediction or racer information is incorrect.On the other hand, it is assumed that for viewing-oriented users, the accuracy of information is not as important as for predictingoriented users.Furthermore, viewing-oriented user's utilities are more subjective and their information needs are more varied than those of predicting-oriented users, and they have a higher demand for diverse text generation.Therefore, as a first step in applying LLM in this study, it is appropriate to apply it to text generation for viewing-oriented users.The first favorite to win is 7-2 with a hit probability of 49.0%. 2 ○Boss, who has the largest quinella rate among the contestants with 95.7% in the last four months; this racer will come out at the top in the last half lap.Subsequently, 7 ○Gretzer will overtake him from behind 2 ○Boss to take the first place.The probability of 2 ○ Boss reversing (2-7) is 25.2%.The dark horse is 7-3.With a hit probability of 3.5% and odds of 30.3, the expected payout is high.○Tamamura will dominate the top of the list.The second favorite to win is 2-9 with a hit probability of 9.2%.Be wary of 2 ○Kasamatsu who boast the highest number of back stretch passes(11) among the contestants and 9 ○Takahashi.Table 3: Examples of text for viewing-oriented users and each evaluator's score ("Is the information valuable to viewing-oriented users?").eval  shows the evaluation value answered by evaluator id .
text eval 0 eval 1 eval 2 eval 3 eval 4 On October 6th, one day later, 9 ○Takakura will be celebrating his 32nd birthday.Although he finished 6th in his last race, he has a chance for revenge, since he has the highest predicted probability of winning 27.3% in today's race.In middle school, he played baseball and in high school, he played in bicycle races.His favorite food is curry and least favorite food is celery.(ja)"1日後の10月6日が32歳の誕生日となる 9 ○高倉。前回のレースでは6着と残念な結果 だったが、今日のレースでの予測勝率は最も高い27.3%で前回のリベンジのチャンス。中 学時代は野球、高校時代は自転車競技をしていた。好きな食べ物はカレー、嫌いな食べ 物はセロリ。" On September 18th, the other day, 4 ○Ohnishi was celebrating his 26th birthday.He finished 6th in his last race.Also, his predicted win probability for today's race is not high, ranking 6th at 7.5%.He has had a tough run of races, but this is where he needs to give his best.In middle school, he played table tennis and in high school, he played in bicycle races.His favorite food is meat and fish and least favorite food is bitter melon.

Figure 1 :
Figure 1: Overview of decision-making support by presenting information

Figure 2 :
Figure 2: Overview of the text generation process.

Table 1 :
Mean and median scores for each question.

Table 2 :
Examples of text for predicting-oriented users and each evaluator's score ("Is the information valuable to predictingoriented users?").eval  shows the evaluation value answered by evaluator id .