Trending Now: Modeling Trend Recommendations

Modern recommender systems usually include separate recommendation carousels such as ‘trending now’ to list trending items and further boost their popularity, thereby attracting active users. Though widely useful, such ‘trending now’ carousels typically generate item lists based on simple heuristics, e.g., the number of interactions within a time interval, and therefore still leave much room for improvement. This paper aims to systematically study this under-explored but important problem from the new perspective of time series forecasting. We first provide a set of rigorous definitions related to item trendiness and formulate the trend recommendation task as a one-step time series forecasting problem. We then propose a deep latent variable model, dubbed Trend Recommender (TrendRec), to forecast items’ future trends and generate trending item lists. Furthermore, we design associated evaluation protocols for trend recommendation. Experiments on real-world datasets from various domains show that our TrendRec significantly outperforms the baselines, verifying our model’s effectiveness.


INTRODUCTION
Nowadays, recommender systems (RecSys) have been widely adopted in real-world scenarios [12,13,15,19,33,40].It is common to design a variety of recommendation carousels on the homepages of websites or mobile apps to explore and capture different perspectives of user interests, such as 'recommended for you' [24], 'buy it again' [7], 'frequently bought together' [10,37], etc.In this work, we study an important yet under-explored class of recommendation carousels called 'trending now', which identify the trends at the current moment and promote trending items to the users.
Recommender systems are known to suffer from the 'rich-getricher' problem, where popular items are more likely to be recommended [1,34].While the global or recent popular items are frequently captured by recommendation carousels such as 'recommended for you' (due to popularity bias [41,43]) and 'recent popular', we hope the 'trending now' carousel can provide a complementary perspective by promoting items that are not frequently appearing in other carousels, but are rising fast in popularity and have the potential to become popular in the near future.
Despite its significance in practice, there is surprisingly limited previous research on modeling trend in the recommendation context.Existing related works [2,4,23,26,42] focus on either modeling trend to improve personalized recommendation or identifying trends in a non-recommendation scenario such as social media; none of them focus on recommending trending items.To bridge this gap, in this paper, we formally define the notion of 'trend', formulate the trend recommendation task for our specific use case, and develop new models tailored for this task.
Specifically, we first define popularity as the number of interactions at a certain time interval.We then define 'trend' in the recommendation context as the change rate of popularity, or acceleration.In line with the definitions above, the primary target of the 'trending now' carousel is to identify and promote items that are receiving increasingly more interactions at the current moment, while these items are not necessarily the global or recent most popular items.This can also be viewed as conducting an effective item exploration without popularity bias.In fact, through our data analysis, we observed a relatively low overlap between the popular items and the trending items based on our definition.
It is desirable to capture the current trend instantly and recommend trending items in real time.However, in reality, a certain amount of time is required to collect sufficient interactions in order to reliably and stably identify the current trends, while the trends are changing dynamically and may drift during data collection period.Given this challenge, we formulate the trend recommendation task as a one-step forecasting problem.Fig. 1(left) demonstrates our problem setting: given items' historical accelerations, we aim to predict which items will be trendy in a short future time interval, i.e., the next time step.This formulation is closely aligned with online real-world scenarios: once the model predicts trending items at the next time step, we display them to users throughout the next time step while buffering data at the back-end.At the end of the next time step, the model makes new predictions for the time step right after it based on newly accumulated data.We then refresh the recommendations and start another cycle of data collection.So, how should we design a trend recommender that is optimized under our specific use case?Naturally, one could build on existing time series forecasting models.There is a rich literature on this topic.Among them, the state of the art is sequential deep learning, which adaptively learns how to effectively ingest historical time series as context and predicts future [18,20,22,28,31,35,39].
On top of time series forecasting models, the unique property of the recommendation context offers interesting new opportunities for adapting these models in our scenario.Specifically, beyond just coarse cumulative numbers of interactions for each item in the time series, there exist more fine-grained user-item interactions in our setting.In other words, for a given item, we not only know how many users have interacted with it, but also know who exactly these users are.This brings additional information to help us discover the underlying correlations between items; items with a large number of overlapping users may share common trend patterns, and therefore improve the trend forecasting accuracy.Fig. 1(right) illustrates our modeling motivation.
In this paper, we design a principled model, dubbed TrendRec, for trend recommendation.Our model can be instantiated with any deep learning based time series forecasting models, and is capable of leveraging user-item interactions as additional context.Tren-dRec is a two-phase framework and is trained step-by-step on both the auxiliary objective of next item recommendation as well as the primary time series forecasting objective of next-step acceleration prediction.Specifically, through initial training on the next item recommendation objective, TrendRec leverages user-item interactive signals to detect the underlying correlations across items and encode such knowledge as item embeddings.In the second phase training, the learned item embeddings are used to provide additional context on the respective time series for the time series forecasting objective.Our main contributions are summarized as follows: • We formally define the notion of 'trend' in the recommendation context, and establish corresponding evaluation metrics and evaluation process.

PRELIMINARIES
In this section we first introduce trend-related terms and trend definition, followed by our formal definition of trend recommendation problem, and then we introduce some basic models which will be used as baselines in the experiment section.

Defining Acceleration as Trend
We first define the following terms that will be used in the rest of the paper: (1) Trend Definition.For an item , we define its trend at time step  to be its acceleration at time step , which is A  .An item  is considered as trendy at time step  if its acceleration A  is among the highest of all items' accelerations at time step .In the following part, the terms 'acceleration' and 'trend' will be used interchangeably as they are equivalent in our definition.

Why define acceleration as trend in the recommendation regime?
The trending now carousel is expected to recommend a complementary group of items, which rarely overlap with items recommended by other carousels, yet are receiving increasingly more interactions.These trending up items are not necessarily the global or recent most popular items.Recommending items with large accelerations allows us to conduct an effective item exploration on promising trending up items that may not be popular yet, raising their exposure rate to further facilitate their trends.Some exemplar types of targeting items are as follows: • Recently released cold items of good quality (e.g., a new episode of Game of Thrones).• Items experiencing sudden changes (e.g., a movie won an Oscar award and abruptly becomes trendy).• Long lasting items with periodic up-trend (e.g., items with seasonal effect such as winter clothes).

Problem Definition
Ideally, we promptly detect the current trend and recommend trending items in real time.Nevertheless, in practice, a certain amount of time is necessary for accumulating adequate interactions to ensure reliable identification of the current trend, while trends are subject to dynamic variations and may drift during the course of data collection.In other words, we hope the time interval Δ to be small enough so that A  reflects the trend at the current moment, but with a small Δ, A  can be noisy due to insufficient data collected during Δ.Therefore, we argue that instead of capturing and surfacing trending items in real-time, it is more feasible and reasonable to predict and display trending items in a short future time interval, while frequently update our recommendations based on the newly collected data.
Based on the observation above, we define the trend recommendation task as a one-step time series forecasting problem.Formally, for each item, given its historical acceleration [A 0 , A 1 , . . ., A  ] := A ,0: and additional contextual information [C 0 , C 1 , . . ., C  ] := C ,0: such as covariates [31], we hope to predict its acceleration at the next time step  + 1 and we recommend the top  items based on the trend prediction.Hypothesis on correlations between the time step length and the task feasibility.It is worth noting that different lengths of time steps suggest different sets of ground-truth items (e.g., whether to predict the top  items with highest acceleration in the next one hour or in the next one day), and thus will be mapped to different problems of different difficulty levels.The correlations between the task feasibility and the time step length is postulated as the curve shown in Fig. 2, which essentially depicts a bias-variance tradeoff: when the time step length is small (e.g., one hour), variance is introduced due to data sparsity; when the time step length is large (e.g., one day), bias is introduced due to temporal drift.Therefore, we need to find a 'sweet spot' that strikes a good balance between the two.
Later in Sec.4.3 we propose two evaluation metrics for acceleration which are closely aligned with our trendiness definition and the problem formulation for trend recommendation.The above  Figure 2: Hypothesis on the correlation between the task feasibility and the time step length Δ.When Δ is small, the trend can be noisy due to insufficient collected data, introducing variance; when Δ is large, the trend may already changed due to the long process of data collection, leading to bias.We therefore propose a bias-variance tradeoff over the choice of Δ with the goal of finding a balance.This hypothesis is verified by experimental results in Sec.4.5.
hypothesis is validated by experimental results based on the proposed metrics (refer to Sec. 4.5 for more details) on a variety of datasets across different domains.We further develop an adaptive mechanism to select the most appropriate time interval Δ for each dataset.

Baseline Models
Here we present various basic models for trend recommendation.Based on the problem definition in Sec.2.2, it is natural to leverage existing methods designed for the time series forecasting task.We first describe two widely adopted heuristic models and then introduce a generic form of deep learning based probabilistic time series forecasting models.All models introduced here will be used as baselines in the experiment section.

Markov Heuristic Model.
we assume an arbitrary item 's acceleration at the next time step A  ( +1) only depends on its acceleration at the current time step A  .Since in reality the accelerations are prone to remain the same over short period of time, we define the Markov heuristic model as: where Â ( +1) is the predicted acceleration at the next time step.Note that the Markov heuristic model is essentially a special case of AR [9].

Exponential
Moving Average (EMA) Heuristic Model.The main drawback of the Markov heuristic model is that the item acceleration at the next time step is exclusively influenced by the current time step, which can be noisy due to issues such as data sparsity.We therefore hope to take multiple latest time steps into consideration, assigning more weights to the more recent time steps.This leads us to define the Exponential Moving Average (EMA) heuristic model as below: where Â ( +1) is the predicted acceleration at the next time step,  is the number of recent time steps the model is considering, and   is the predefined weight which decays exponentially with the increasing number of steps away from the current time step.Note that the EMA model can be viewed as a special case of ARIMA [8].

Deep
Learning based Time Series Forecasting Model.The heuristic models typically encode general assumptions that lack the flexibility to adapt to diverse scenarios.However, the acceleration patterns vary across domains (retail, media, news, etc.).For example, there exist abundant periodical acceleration patterns of different cadences (daily, weekly, seasonal, etc.) in both retail and media domains, such as a new episode of a TV series is released every Wednesday.On the contrary, such regular acceleration patterns are rarely observed in the news domain, as news are time-sensitive and people tend to follow most recent news.Additionally, even within the same domain, various acceleration patterns may coexist.
For example, on a specific movie platform, the acceleration curve of newly released action movies may consistently exhibit a steeper increase compared to that of newly released documentary movies due to the preference of the user community on that platform.A more general solution for trend recommendation is therefore to design a learnable deep learning based time series forecasting model that is adaptive to a variety of scenarios.We formulate the model as: where  seq (•) is a sequential model that aggregates the historical accelerations and projects the probabilistic distribution of the acceleration at the next time step.There is a rich body of literature under the topic of deep learning based sequential time series forecasting which adopted different sequential models  seq (•), such as DeepAR [31] based on RNN, MQCNN [35] based on CNN, TFT [22] based on transformer, and so on.

TRENDREC: COLLABORATIVE TIME SERIES FORECASTING MODEL WITH USER-ITEM INTERACTIONS
In this section, we present our TrendRec, a principled two-phase model which detects item correlations from user-item interactions to facilitate the trend forecasting.It is compatible with any deep learning based time series forecasting models.

Model Overview
Unlike typical time series forecasting scenarios where the input data is one or multiple series of values (e.g., a time series of daily product consumption of a company) [22,31,35], the unique recommendation context promises more fine-grained user-item interactions instead of coarse cumulative number of interactions for each item at each time step.Such user-item interactions provide extra information beyond the number of interactions of an item, as they also reveal the exact group of users who interacted with the item.It is a reasonable assumption that items with lots of overlapping interactive users are likely to have common item properties and targeting users, therefore possess similar trend (acceleration) patterns.For example, 'The Avengers' and 'Justice League' are both superhero movies that appeal to a similar audience, and have a significant number of overlapping viewers.As a result, they have very similar trends: a strong opening weekend as their fan base rushes to watch them, followed by a gradual decline in the following weeks.
To distill knowledge from user-item interactions, we design an auxiliary training objective on next item recommendation task.This objective facilitates representation learning on item properties and yields dense latent item embeddings.The generated latent item embeddings envelop underlying correlations between items which are implied by user-item interactions, and can be employed to facilitate the time series forecasting objective by providing additional context for the corresponding time series.
Overall, TrendRec encompasses two components: a recommendation model and a time series forecasting model.The recommendation model is trained on the next item recommendation objective with user-item interactions.The time series forecasting model is trained on the next-step acceleration prediction objective with time series of items' accelerations.The two components are connected through the shared latent item embeddings which encode item correlations.
Fig. 3 shows the probabilistic graphical model (PGM) for Tren-dRec, which is partially inspired by collaborative deep learning (CDL) [33] and ZESRec [12].It integrates two training objectives: (1) next item recommendation (right part), and (2) item acceleration forecasting (left part).Below we explain its rationale in detail: Figure 3: The probabilistic graphical model (PGM) for our TrendRec.Overall, U  , V  are the hidden variables,   and   are the hyperparameters related to the distribution variance of U  and V  , respectively, while S  , R   , A  ( +1) , and A ,0: are the observed variables.Specifically, U  and S  represent user 's interests till time step  and user 's historical interaction sequence till time step , respectively.V  and A ,0: represent item 's properties till time step  and item 's historical accelerations till time step , respectively.R   is the interaction label denoting whether user  interacted with item  at time step , and A  ( +1) is the item 's acceleration at the next time step  + 1.Time step  represents the time interval [ • Δ, ( + 1) • Δ]. ,  ,  denote number of users, number of items, and number of time steps, respectively.Note that we define P(U  |S  ,   ) = N (U  ;  seq (S  ),  −1  I  )) and P(A  ( +1) |V  , A ,0: ) =  ts (V  , A ,0: ), and both  seq (•) and  ts (•) are learnable functions.For simplicity we exclude their corresponding learnable parameters in the PGM.series forecasting model.The PGM for classic sequential RecSys models such as [15,19,32] is essentially the subgraph of Fig. 3 with nodes   ,   ,    ,   ,   , and   .The PGM for classic sequential time series forecasting models such as [22,31,35] is essentially the subgraph of Fig. 3 with nodes A ,0: and A  ( +1) .
Generative Process.Inspired by CDL [33] and ZESRec [12], the generative process in Fig. 3 is defined as follows: For each time step  ∈ [ ]: (a) For each item  ∈ [ ]: • Draw a latent item offset vector    ∼ N 0,  −1  I  .• Adopt the latent item offset vector as the latent item embedding: • Draw a latent user offset vector    ∼ N 0,  −1  I  .• Obtain the user embedding: n  =  seq (S  ).
• Compute the latent user embedding: For user  we compute the recommendation scores for all items: R  *  ∼  ( [Y   ]  =1 ), where ' * ' represents the collection of all elements in a specific dimension, and  (•) denotes a categorical distribution.
Here  ,  ,  denote number of users, number of items, and number of time steps, respectively. softmax (•) is the softmax function, and ). N (x; ,  −1 I  ) denotes the probability density function (PDF) of a Gaussian distribution with mean  and diagonal covariance  −1 I  for the variable x.   (•) represents any type of probabilistic time series forecasting model which consumes both item historical accelerations and latent item embeddings to predict the probabilistic distribution of acceleration at the next time step  +1. seq (•) denotes any type of sequential recommender system which generates the user embedding based on user's historical interactions, and in this work we adopt the classic recommender system, GRU4Rec [15].

Training
Maximum a Posteriori (MAP) Estimation.Given the graphical model in Fig. 3, we hope to estimate the latent variables U  and V  as well as the learnable parameters in functions  seq (•) and  ts (•).Note that for simplicity, we exclude the learnable parameters of  seq (•) and  ts (•) in the PGM.The maximum a posteriori (MAP) estimation can be decomposed as follows: ( Below we explain each conditional probability from Eqn. 5 in detail.
We assume Gaussian distributions on latent variables U  and V  in Fig. 3 as follows: For the next item recommendation, we define the conditional probability over the observed interactions as:  For the time series forecasting, in line with Eqn. 4, we define the conditional probability over the observed item accelerations as: Negative Log Likelihood (NLL).Maximizing the posterior probability is equivalent to minimizing the negative log likelihood (NLL) of P(U  , V  |R   , A  ( +1) , S  , A ,0: ,   ,   , ).We substitute each terms in Eqn. 5 with our definitions in Eqn.6, Eqn. 7, Eqn. 8, and Eqn. 9, and compute the NLL as: where  seq (S  ) = n  , and ℓ (•) represents a likelihood function adopted by the probabilistic time series forecasting model.Note that we train hidden variables U it and V jt as well as the function  seq (•) and  ts (•) based on the NLL training objective above.Below we describe the intuition of each term in L.
( Embedding U  (Eqn.12).The first term regularizes the latent item embedding V  to be close to the zero-mean Gaussian prior.The second term regularizes the latent user embedding U  to be close to the computed  seq (S  ) as user history implies user interests, while providing flexibility for U  to deviate from  seq (S  ) as users with the same history may still have non-overlapping interests.

Inference
During inference, we only focus on the time series forecasting task and predict the acceleration as below: where with a bit overload on * , we use V *  to denote the posterior of item 's latent item embedding, and  * ts (•) to denote the trained sequential time series forecasting model.Next Item Recommendation Objective.For the next item recommendation objective, TrendRec generates the latent user embedding from a sequential recommender, and adopts latent item offset vector as the latent item embedding.The recommendation score is calculated based on the inner product between the latent user embedding and the latent item embedding, which is subsequently normalized by the softmax function  softmax (•).

Model Architecture
Time Series Forecasting Objective.For the time series forecasting objective, for each item, TrendRec retrieves the latent item embedding pre-trained on the next item recommendation objective and utilizes it in conjunction with the item historical accelerations to predict the acceleration at the next time step.
In the experiment, for simplicity, we remove the regularization term on the latent user embedding, and decouple the training process by initially training on the next item recommendation (NIR) objective (Eqn.14), followed by training on the time series forecasting (TSF) objective (Eqn.15) which is our primary target: where V *  denotes the posterior of item 's latent item embedding after training on the NIR objective.

Practical Challenges and Solutions
We describe the unique practical impediments on adopting the time series forecasting models in the trend recommendation scenario, and propose various mechanisms to alleviate them.
Skewed Data.A real-world recommendation dataset usually includes hundreds of thousands of items.Due to the well known 'heavy-tail' problem in the recommendation regime, the majority of items are considered as cold items owing to low number of interactions (e.g., less than five interactions).As a result, the time series associated with these items exhibit nearly flat curves, hovering around zero.The time series forecasting models can be biased towards predicting zeros if a substantial proportion of such time series exist in the training data.Short Active Period.The time series of warm items may still suffer from short active periods due to reasons below: • Varying active periods across items: From data analysis, we found it is pervasive that the item time series contain leading zeros (because the item is not yet released) and trailing zeros (because the item is no longer available).This is primarily caused by the mismatch between the active periods of different items.For example, item A might be introduced to the market just as item B becomes unavailable.• Short lifecycle: Items in certain domains have short lifecycles by their nature.For example, in the news domain, the lifecycle of an item normally spans less than one day as most people only pay attention to today's news.
Solutions.The aforementioned issues essentially pertain to data sparsity, and a principled solution is to filter out noise in the training data to avoid the garbage-in-garbage-out conundrum.In practice, we inspect the acceleration curve for each item and select top % of items that display the most dynamic changes in acceleration, where  is a tunable hyperparameter.The dynamic nature of acceleration for an item  is measured with the metric D  =   |A  |.We further adopt training masks on the time series of selected items; this enforces the model to focus on the active period and updates gradients based on the non-zero values.Additionally, we implement a range of mechanisms aimed at improving training stability and inference latency, please refer to Appendix A for more details.

EXPERIMENTS
We evaluate the heuristic models, time series forecasting models, and the proposed TrendRec on three public datasets from a variety of domains including retail, media, and news, with the major goals of addressing the following questions: Q1 Does the proposed hypothesis on correlations between the task feasibility and the time step length in Sec.2.2 (see Fig. 2) hold for all datasets?How do we select the time step length for each dataset?Q2 Does TrendRec outperform all the baseline models including the heuristic models and the vanilla deep learning based time series forecasting model?
(1) TaoBao User Behavior [44] is a public dataset containing user behavior data from Taobao, one of the largest e-commerce platforms in the world.It contains around 1M users, 4M items, and covers 9 days of interaction data.Due to the large item catalogue, in the experiment, we split the data by item category, and select the top-3 item categories based on number of interactions to construct three separate datasets.We call them TaoBao Cat 1, TaoBao Cat 2, and TaoBao Cat 3, respectively.(2) Netflix [6] is a classical recommendation dataset including 100M user ratings on movies with 17K items and 480K users.We consider the densest time period from 2003 January to 2005 December for our experiment.(3) MIND [36] is a large-scale news recommendation dataset collected from the user interaction logs of Microsoft News.It includes 1-week user-item interactions and 4-week impression logs on 1M users and 160K news articles.In the experiment, we only utilize the 1-week interactions as the impression logs do not include timestamp information.The statistics of the datasets after pre-processing are shown in Table 1.We adopted a rigorous experimental setup to ensure there is no temporal leakage between the next item recommendation objective and the time series forecasting objective.Specifically, we (1) temporally split the data such that all training interactions must happen before all testing interactions and (2) use the exact same training data for both objectives during the training phase.

Evaluated Methods
We consider the following methods during evaluation: • Oracle, which has access to the ground truth future acceleration at the next time step.Therefore, it always accurately predicts the acceleration and recommends the correct top- trending items.2) DeepAR is a likelihood method, and therefore it is more suitable for computing MAP.For the next item recommendation objective, we adopt GRU4Rec [15] to learn the latent item embeddings.

Evaluation Metrics
Instead of adopting evaluation metrics such as RMSE from the time series forecasting setting, we design evaluation metrics that are closely aligned with the goal of trending now recommendation carousel, which is to promote trending items at the next time step.Different from typical time series forecasting settings, for trend recommendation we are only interested in correctly recommending the top- trending items instead of accurately predicting the acceleration for each item.Therefore, we design the metrics with emphasis on measuring the model's ability of identifying the items with large acceleration at the next time step.Similar to the classical evaluation metrics, Recall and NDCG, which are widely adopted in the recommendation domain, we relate acceleration to relevance and propose two evaluation metrics: Acceleration (Acc) and TNDCG.
Assuming the current time step is  − 1, and we hope to evaluate on the time step , we define the proposed metrics as follows.

Acceleration Metric.
We first select the top- items based on the model's predicted accelerations at the next time step , and denote them as P. Then we map the selected  items to their corresponding ground truth accelerations at the next time step  as {A  }  ∈ P .Since in our definition, acceleration is a quantitative measurement of trendiness, we compute the sum of next time step accelerations of the predicted items  ∈ P A  and use it as the model's trendiness score.The higher the trendiness score, the better the model at predicting the trending items at the next time step.We hope to scale the metric in the range of [0, 1], and therefore apply the min-max normalization on top of the trendiness score.The upper bound of the trendiness score comes from the Oracle model: we denote the ground truth top- items with largest accelerations at time step  as O, and compute the trendiness score as   oracle,t =  ∈ O A  .There is no lower bound for the trendiness score as the acceleration can be an arbitrary negative value.We artificially set the lower bound as the trendiness score of the Random model.This also helps prevent over-penalization on poor recommendations at a single time step, as the model is evaluated on multiple time steps (see Sec. ).Finally, we combine different trendiness scores to compute the acceleration metric at time step : where @  ∈ [0, 1].

TNDCG Metric.
Our Trendiness-Normalized-DCG (TNDCG) metric further takes the item's rank position into consideration with a logarithmic reduction factor.We use  to index the rank position, A P  and A O  to represent the acceleration of the item ranked at position  based on order from model prediction and ground truth, respectively.Then we compute TDCG  model, =   =1 +1) .We use the min-max normalization to scale the final TNDCG metric at time step  to the range of [0, 1] and TNDCG@  ∈ [0, 1].

Evaluation Protocol
During evaluation, we split the training and test sets by timestamp, and leave the latest 20% time span for testing (e.g., eight-hour training window, two-hour testing window).The evaluation is conducted in a rolling fashion with a total of  steps.For instance, the test time span is two days while the time step length Δ is six hours; we therefore have  = 8 evaluation steps.
After evaluation, we will have  acceleration metrics for the evaluated model, Random, and Oracle, respectively.We aggregated them through a weighted sum where Acc@ ∈ [0, 1].

Hypothesis Validation (Q1)
We conduct experiments to validate the hypothesis proposed in Fig. 2, which presumes a correlation curve between the task feasibility and the time step length Δ.Specifically, we evaluate the performance of the Markov heuristic model, which simply projects A  ( +1) = A  , on a range of time intervals based on the acceleration metric and plot the curve.The Markov heuristic model is a rudimentary model grounded on a simple but generic assumption, and therefore its performance should reflect the task feasibility.
The results are shown in Fig. 5. On TaoBao and MIND datasets, the curves exhibit a clear trajectory that is consistent with our conjecture, where the acceleration metric first improves with increasingly longer time interval due to alleviation on the data sparsity, and then decreases due to temporal drift.On the other hand, the curve on the Netflix dataset keeps decreasing.This is because the timestamp granularity of the Netflix dataset is one day, which is long enough to collect sufficient data but starts to suffer from the temporal drift.
Overall, the results verify our hypothesis.In the following experiments, we select the time interval Δ for each dataset based on the peak of respective curves: three-hour time interval for all three TaoBao datasets for consistency, one-day time interval for Netflix, and thirty-minute time interval for MIND.

Experimental Results (Q2)
We evaluate our proposed TrendRec against various baselines on datasets from three domains: (1) TaoBao from the Retail domain, (2) Netflix from the Media domain, and (3) MIND from the News domain.The results are summarized in Table 2, from which we can easily conclude that TrendRec demonstrates superior performance.We have the following observations: • TrendRec outperforms all baseline models.Our Tren-dRec consistently shows better performance compared to the baseline models across all the datasets.Since TrendRec's time series forecasting model is instantiated with DeepAR, the performance gain over DeepAR demonstrates the effectiveness of leveraging pre-trained latent item embeddings to boost the time series forecasting performance.• Performance improvement from TrendRec is associated with the quality of latent item embeddings.While Tren-dRec demonstrates the best performance on all cases, we observe that the improvement over DeepAR is most significant on TaoBao, while it is marginal on Netflix and MIND.We present TrendRec's results on the auxiliary objective of next item recommendation in Table 3, where TrendRec shows significantly better results on TaoBao compared to Netflix and MIND.Based on this observation, we argue that the performance gain on the time series forecasting objective, which comes from leveraging pre-trained latent item embeddings, depends on the embedding quality.Notably, this quality is positively related to TrendRec's performance on the next item recommendation objective.Analyzing item time series from the News domain reveals that items typically reach their maximum acceleration in a short period of time once they are released, which is followed by a sudden drop.This is primarily due to the time-sensitive nature of news, and poses a great challenge for deep learning based models.As we only focus on top items with large accelerations at each time step, the major goal of deep learning based models is to learn how to accurately forecast the items' acceleration when they are trendy given limited context (historical trends).

RELATED WORK
Trend Modeling.There are limited previous works have been done on the topic of trend modeling in the recommendation context, while the existing works [3,23,25] typically refer to recent popularity as trend, which is velocity in our term.We argue that promoting recent popular items in the 'trending now' recommendation carousel will not only amplify the 'rich-get-richer' problem, but also will lead to redundant recommendations compared with other carousels such as 'recent popular'.Moreover, none of these works focus on recommending trending items.For example, [23] leverages the trend information to probe users' potential interests, which have not been shown in their historical interactions, in order to improve next item recommendation for the users.Another line of research [2,26,27,38,42] studies trends in another scenario: social media (e.g., Twitter), and adopts a distinct problem setting from ours.For instance, [26] tries to detect bursty keywords from recent tweets, and then identify trending topics based on them.Overall, the previous works in trend modeling are either focusing on a different task or set in a distinct scenario.Therefore their proposed methods are not adaptable to our use case of trend recommendation.Time Series Forecasting.Time series forecasting goes back to several decades in statistical approach including ARIMA [8], ETS [16].However, neural networks showed little success in the forecasting literature until recently.With the explosive sample of time series data available and advances in neural architectures, deep learning for forecasting has become increasingly popular, which includes the architectures of RNN [30,31,35], CNN [20,35], Transformer [20,22,39].Most of these methods support probabilistic forecasting in either distribution-assumed manner as a likelihood approach [20,29,31] or distribution-free manner with quantile regression [18,22,28,35].Multivariate (or local) modelings like DeepVAR [30] are often preferred due to its capacity to encode the correlation between time series or items, but global modeling like DeepAR [31] or MQ-R(C)NN [35] are still widely used by leveraging data samples efficiently.See [17] for more detailed review on modern deep forecasting.
Adopting Time Series Models in Recommendation.Various prior research [5,11,14,21,45] has explored adopting time series models to model the temporal information in the recommendation scenario, with the primary goal of improving next item recommendation performance.STEN [21] employs temporal point processes to model the impact of friends' behaviors on users' dynamic interests.LSHP [11] considers Hawkes Process to model the long-term and short-term dependency between users' online interactive behaviors.Different from the aforementioned works, our main focus lies in accurately predicting trending items in the near future.Additionally, we design an auxiliary training objective to distill item correlations from user-item interactions, thereby boosting our forecasting performance.

CONCLUSION
In this work, we study an under-explored topic in the recommendation regime: the trend recommender.Since limited previous works have been done on this topic, we start by formally defining the notion of 'trend' within the recommendation context.We then recognize a bias-variance tradeoff hindering the model's ability to identify the trend in a timely and stable manner.This tradeoff comprises variance resulting from data sparsity, and bias emerging from temporal drift.Based on this observation, we formulate the trend recommendation task as a one-step time series forecasting problem.In terms of methodology, we develop a principled twophase model, dubbed TrendRec, which harnesses the user-item interactive signals to uncover the underlying correlations across items and ingests such knowledge to facilitate the trend forecasting.We further establish corresponding evaluation protocols for trend recommendation.Extensive experiments on datasets from various domains including retail, media, and news demonstrate the effectiveness of our proposed TrendRec.

A SOLUTIONS TO PRACTICAL CHALLENGES
To address the challenges identified in Sec.3.5, i.e., skewed data and short active period, along with additional empirical problems, we design the following mechanisms in featurization, training, and inference:

A.1 Featurization
• Dynamic Filter: To alleviate the problem of skewed data, we examine acceleration patterns of each item and select top % of items that exhibit the most significant changes in acceleration.
Here  is a tunable hyperparameter.We quantify the dynamics of acceleration for an item  based on the metric D  =   |A  |.Note that this mechanism is closely aligned with the objective of trend recommendation, which is to correctly predict top- trending items instead of accurately forecasting the accelerations of all items at the next time step.

A.3 Inference
• Candidate Filter: To guarantee the quality of recommendations, we apply a velocity filter to select the top % of items as candidates for the acceleration prediction of the time series forecasting model.This filter is based on the velocity of the item at the current time step, and the value of  is a hyperparameter that depends on the dataset.• Active Filter: We exclude the items with zero velocity at the current time step from the candidate list.This is mostly useful for items in the news domain due to short active period.
Both filtering mechanisms reduce the number of candidates prior to the time series forecasting model's inference, with the goal of improving forecasting accuracy and reducing inference latency.

Figure 1 :
Figure 1: Left: Our problem definition on trend recommendation, where given items' historical accelerations, we predict their accelerations at the next time step and rank them accordingly for trend recommendation.In this example, the rank order would be [item B, item A].Right: Motivation for TrendRec: two items with large percentage of overlapping users exhibit similar trends.

Figure 4 :
Figure 4: Model architecture for TrendRec from a neural network perspective.The left side figure shows the network structure, and the right side figure illustrates the implementation details of each network component.In the right side figure, blue and orange imply learnable components trained on the next item recommendation objective and learnable components trained on the time series forecasting objective, respectively.Each SEQ block represents a specific time step of a sequential model.Note that both figures are instantiated on a specific user-item pair (user  and item ).Overall, TrendRec encompasses two major components: (1) a sequential recommender, and (2) a collaborative time series forecasting model, and they are connected through the learnable latent item embeddings.

Fig. 4
Fig. 4 shows the model architecture from the neural network point of view.The left side figure demonstrates the overview network structure, while the right side figure visualizes the full details of the TrendRec implementation.In general, the model contains two principal components: (1) a sequential recommender, and (2) a collaborative time series forecasting model.These two components are joined through the learnable latent item embeddings.Below we elaborate on the process in terms of two training objectives: the next item recommendation objective and the time series forecasting objective.Next Item Recommendation Objective.For the next item recommendation objective, TrendRec generates the latent user embedding from a sequential recommender, and adopts latent item offset vector as the latent item embedding.The recommendation score is calculated based on the inner product between the latent user embedding and the latent item embedding, which is subsequently normalized by the softmax function  softmax (•).Time Series Forecasting Objective.For the time series forecasting objective, for each item, TrendRec retrieves the latent item 4.4 for more details).The trendiness score for the Random model is computed as the expectation over the acceleration summation of  items sampled from an uniform distribution   random, =   |  | • , where   =   A  .The final trendiness score for the model is   model, =  (  ∈ P A  ,  random, For an item , we define its number of interactions collected during time step  as its velocity at time step , and denote it as W  ∈ R.Here W  represents item 's popularity per unit time Δ.(3) Acceleration: We denote the acceleration for item  at time step  as A  ∈ R, and compute it asA  = ΔW  = W  − W  ( −1).Here A  represents that item 's velocity is changing ΔW  per unit time Δ.
Time Step: The time step is a time interval with a predefined duration Δ (e.g., one hour).We denote the time step with time interval [ • Δ, ( + 1) • Δ] as time step , where  is the time step index.(2) Velocity:

Time Step Length 𝚫𝐭 Task Feasibility Variance from Data Sparsity Bias from Temporal Drift Enough data, mild temporal drift
∈ {0, 1} is the interaction label denoting whether user  interacted with item  at time step .•Variable A  ( +1) ∈ R is the acceleration of item  at the next time step  + 1. • Edge S  → U  : User's previous interactions reveal the user interests and affect the user's next action (e.g., a user who purchased a cell phone may want to purchase its accessories next).• Edge {U  , V  } → R   : Interaction depends on user interests U  and item properties V  .• Edge {V  , A ,0: } → A  ( +1) : The acceleration of item  at the next time step  +1 is influenced by item's properties (e.g., action movie is more likely to become trendy in the user community of a specific website) and item's historical acceleration (e.g., an item with periodical weekly trend pattern).
item 's properties till time step  (both static properties and dynamic properties) and item 's historical acceleration till time step  which is [A 0 , A 1 , ..., A  ], respectively.denotesthenumber of historical time steps of item  till time step .is the hidden dimension of the embedding.•VariablesU∈ R  and S  ∈ R   × represent user 's interests till time step  and user 's historical interaction sequence till time step , respectively.Here   is the number of interactions from user  till time step .S  is an embedding matrix and each row of it represents an item embedding.•VariableRHere U  , V  are the learnable embeddings with the same hidden dimension , and we term them, latent user embedding and latent item embedding, respectively.  and   , are hyperparameters related to distribution variance.The corresponding conditional probabilities in the PGM are listed in Eqn. 5.In general, TrendRec is an organic combination of a recommender system (RecSys) and a time
• Random, which recommends items by random selection from the whole item catalogue without replacement.• Markov, which is a rule-based model to predict the acceleration at the next time step to be the same as the acceleration at the current time step.See Sec.2.3.1 for more details.• Exponential Moving Average (EMA), which is a rule-based model to predict the acceleration at the next time step based on a weighted sum of accelerations at the latest  time steps (in the experiment we set  = 8).The weight decays exponentially with a factor of 0.75 with increasing number of time steps away from the current time step.See Sec.2.3.2 for more details.• DeepAR [31], which is one of the state-of-the-art time series forecasting models built on an auto-regressive recurrent network (RNN).See Sec.2.3.3 for more details.• TrendRec, which is our proposed two-phase model.It is instantiated on the DeepAR model for the time series forecasting task.The reason we choose DeepAR is twofold: (1) it is one of the state-of-the-art time series forecasting models and has been widely adopted, and ( The weight for each time step is   oracle, −   random, , and   (  oracle, −   random, ) is the partition function for normalization.The intuition behind our weight design is that a time step is more important if the performance gap between Oracle and Random is large at that time step, as there is a large room for improvement.Validating our hypothesis on the correlation between the task feasibility and the time interval (i.e., time step length Δ).Datasets from left to right are TaoBao (all three datasets), Netflix, and MIND.Note that the timestamp granularity of Netflix is one day; hence we are only able to show the right-hand side of the full curve, where model has collected sufficient data but is suffering from the temporal drift.Experimental results verified our proposed hypothesis in Sec.2.2 (see Fig.2).

•
Deep learning based models significantly outperform heuristic models.The deep learning based models (both DeepAR and TrendRec) outperform the heuristic models by a large margin, especially on TaoBao and Netflix datasets.The results emphasize the importance of adopting a learnable model for trend recommendation.• The EMA model is worse than the Markov model in most cases.Interestingly, the EMA model underperforms the Markov model in most cases.This shows that the trend is changing dynamically and that the dependency relationship between the trend at the next time step and historical trends is more complicated than a simple weighted sum with recency bias.• Performance gain from deep learning based models is relatively small in the News domain.The performance gap between the deep learning based models (both DeepAR and TrendRec) and the heuristic models is relatively marginal

Table 2 :
Trend recommendation results on three TaoBao datasets in the retail domain, the Netflix dataset in the media domain, and the MIND dataset in the news domain.Based on the results in Sec.4.5, in experiments we set the time step length Δ to three hours, one day, and thirty minutes for TaoBao (all three datasets for consistency), Netflix, and MIND, respectively.The best results are shown in bold.
in the News domain compared to Retail and Media domains.

•
Training Masks: Due to the problem of short active period, the time series of selected items (i.e., items chosen by the dynamic filter) can remain sparse.To prevent the model from overfitting zeros, we employ training masks which instruct the model to disregard zeros during training and only update gradients based on non-zero values.• Different Context Windows for Different Domains: The context window is a hyperparameter of time series forecasting models which specifies the number of past time steps the model should take into account.Depending on the domain, such as media and retail with longer active periods for items or news with shorter item lifecycles, the context window size is adjusted.For example, a larger context window of 30 time steps may be used for the media and retail domains, while a smaller context window of 2 time steps may be used for the news domain.• Training on Velocity: To ensure training stability, we choose to train the time series forecasting model using velocity time series instead of more dynamic acceleration time series.Empirically, we discovered that the time series forecasting model trained on velocity time series achieved better performance.