CDRNP: Cross-Domain Recommendation to Cold-Start Users via Neural Process

Cross-domain recommendation (CDR) has been proven as a promising way to tackle the user cold-start problem, which aims to make recommendations for users in the target domain by transferring the user preference derived from the source domain. Traditional CDR studies follow the embedding and mapping (EMCDR) paradigm, which transfers user representations from the source to target domain by learning a user-shared mapping function, neglecting the user-specific preference. Recent CDR studies attempt to learn user-specific mapping functions in meta-learning paradigm, which regards each user's CDR as an individual task, but neglects the preference correlations among users, limiting the beneficial information for user representations. Moreover, both of the paradigms neglect the explicit user-item interactions from both domains during the mapping process. To address the above issues, this paper proposes a novel CDR framework with neural process (NP), termed as CDRNP. Particularly, it develops the meta-learning paradigm to leverage user-specific preference, and further introduces a stochastic process by NP to capture the preference correlations among the overlapping and cold-start users, thus generating more powerful mapping functions by mapping the user-specific preference and common preference correlations to a predictive probability distribution. In addition, we also introduce a preference remainer to enhance the common preference from the overlapping users, and finally devises an adaptive conditional decoder with preference modulation to make prediction for cold-start users with items in the target domain. Experimental results demonstrate that CDRNP outperforms previous SOTA methods in three real-world CDR scenarios.


INTRODUCTION
Recommendation systems (RS) have been widely deployed in realworld applications, such as Amazon (E-commerce) and YouTube (online video).As a promising method, collaborative filtering (CF) has been successfully applied to RS, ranging from early matrix factorization methods [26,31] to the recent deep neural network methods [9,35].However, CF-based methods usually suffer from the long-standing challenge of user cold-start problem [39], where the new emerging user has few user-item interactions that can hardly learn effective representations to make recommendation.To address this problem, cross-domain recommendation (CDR) [15,18] has been proposed recently, which explores the cold-start users by utilizing the rich user-item interactions observed from an informative source domain, so as to improve the cold-start users' recommendation in the target domain.
A common idea of CDR methods is to train the model with the overlapping users appearing in both the source and target domain, and then make recommendation for the cold-start users that only have interactions in the source domain.To achieve the above idea, traditional CDR methods [10,20,27,30,39] follow the Embedding and Mapping (EMCDR) paradigm (as shown in Fig. 1(a)).This paradigm aims to learn a user-shared mapping function.For implementation, EMCDR first generates user/item representations for each domain separately, and then trains a mapping function to align the overlapping users representations between the source and target domain.Afterward, given a cold-start user representation from the source domain, EMCDR uses the mapping function to predict the user representations in the target domain.However, such a paradigm learns a unified mapping function shared by all users, neglecting the different user-specific preferences, thus leading to inflexible representation transferring for different users.
Recent studies attempt meta-learning [6,22] to further capture user-specific preference in CDR (as shown in Fig. 1(b)).They mostly regard each user's CDR as an individual task, and train the model to achieve these user-specific tasks with the users in the support set, thus can make recommendation for cold-start users in the query set.Among them, PTUPCDR [40] proposes a meta network with a task-oriented optimization procedure, and generates cold-start user representation by personalized mapping functions in prediction.However, this kind of methods usually regard each user's CDR as an independent task, neglecting the preference correlations among users, where the similar users may have similar preferences which can provide beneficial information for training the mapping functions.Moreover, both of the EMCDR-based and the metalearning-based methods mainly concentrate on the transferring of the learned user representations, neglecting the explicit user-item interactions from both domains during the mapping process, thus leading to sub-optimal user preference modeling.
To address the above issues, we propose a novel meta-learning framework for CDR based on Neural Process [7,8], termed as CDRNP (as shown in Fig. 1(c)).Specifically, we build our model with Neural Process (NP) principle, which formulates the CDR task of all users within the same stochastic process, naturally capturing the preference correlations among the overlapping and cold-start users.Besides, the model also follows the meta-learning paradigm, which not only preserves user-specific information for each user, but also utilizes explicit user-item interactions from both domains beyond the sole user representation transferring.Such a design enables CDRNP to learn more representative user-specific mapping functions and achieve bi-directional recommendation with same observed data.For implementation of the CDRNP framework, we first propose a characteristic embedding layer which leverages each user's interactions observed in the source domain to generate its user-specific preference representation, and then combine it with the candidate item in the target domain, constituting the complete rating-observed information as input for each candidate item rating.Then, we propose a preference aggregator to approximate CDR for each user as a particular instance of a stochastic process, which captures common preference correlations among the cold-start users and overlapping users.Additionally, we devise a preference remainer to enhance the common preference representations from the overlapping users.Finally, we devise an adaptive conditional decoder to modulate both preference representations to make better rating prediction for the cold-start users in the target domain.
In summary, the contributions of this paper are as follows: • To our best knowledge, we introduce a fresh perspective to solve the cold-start problem in CDR with NP principle.• We propose a novel meta-learning CDR method with NP, termed as CDRNP, which preserves the capability of user-specific mapping function by meta-learning paradigm, while further captures preference correlations among users with NP. • Extensive experiments on three CDR scenarios demonstrate that CDRNP outperforms the state-of-the-art methods.

PRELIMINARY
This section introduces the CDR problem settings and the background knowledge of the neural process principle.

Problem Setting
This paper considers a general CDR scenario: given the observed user-item interactions of both domains, we aim to make recommendations in the target domain for cold-start users, with the help of user-item interactions in the source domain.Formally, the data can be formulated as D / = (U / , V / , Y / ), where the superscript / denotes the source/target domain, and U, V and Y ∈ {0, 1, 2, 3, 4, 5} |U | × |V | denote the user set, item set and useritem rating matrix, respectively.Here the rating score  (, ) ∈ Y ranges from 0 to 5, reflecting the preference tendency of user  ∈ U to choose item  ∈ V. Further, we define the users overlapped in both domains as the overlapping users U  = U  ∩ U  , then the cold-start users from the source domain can be denoted as U \ = U  \ U  .Therefore, for each cold-start user  \ ∈ U \ , given its interactions from the source domain V  ( \ ), the goal is to predict the rating  ( \ ,   ) for each candidate   ∈ V  .
To achieve the above goal, we develop our model with metalearning settings [1,3,13,19,33]  ∈ U \ in Q  .In such a way, the model learns common knowledge of achieving CDR in the training phase, thus can make recommendation for the cold-start users to achieve CDR in the testing phase.

Neural Process Principle
To achieve CDR, we develop our model based on Neural Process (NP) [8].Generally, NP is a kind of neural latent variable models, which approximates stochastic processes parameterized by powerful neural networks.In this paper, we assume each task associated with an instantiation of the stochastic process   , and each pair (  ,    ) can be seen as a sample point drawn from   .Therefore, the corresponding probability distribution  of   can be defined as follows: where  1:  and   1:  denote the all   and    decoupled from   , respectively, and we use   = |  | for simplicity.Following NP, we approximate the stochastic process   with a low-dimensional random vector   generated by non-linear neural networks.Then, with ...condition on tasks, we rewrite the generation process as: Here we assume  (  ) ∼ N (0,  ) for simplicity in paradigmatic variational frameworks [12], and then we can easily sample   from  (  ) to achieve concrete function implementation.Since the true posterior distribution is intractable, we adopt amortised variational inference [21] Considering that each task   contains a support set C  and a query set Q  , we would like to make predictions for Q  given C  .Therefore, we rewrite Eq. ( 3) as follows: where Eq. ( 5) is the lower bound of Eq. ( 4) with derivation like Eq. ( 3).In practice,  (  |C  ) in Eq. ( 5) is intractable, thus we use variational prior (  |C  ) to approximate  (  |C  ).Such an approximation can be regarded as a regularization term of KL divergence to ensure the  condition of stochastic process [24].Thereby, we summarize Eq. ( 5) into Eq.( 6), which can be intuitively explained with two terms: the first term aims to achieve the desired CDR task, and the second term aims to ensure all tasks in the query and support set following the same stochastic process.
For implementation, we illustrate the above principle from a practical perspective.Fig. 2 shows our model with NP in the training and testing phase.Note that in the training phase, we use L  to assimilate (  |Q  ) and (  |C  ), which learns   general to both the support and query set in each task.In the testing phase, we use   generated from the (  |C  ) to substitute for (  |Q  ), and thus together with =1 in the given task, we make predictions for the users by  (|, ).In this way1 , we encourage the tasks of users following the same stochastic process, thus naturally captures the preference correlations among overlapping and cold-start users.

MODEL
This section introduces our model CDRNP implementing the NP principle.Fig. 3 sketches the overall framework of CDRNP.Our model consists of three components: 1) A characteristic embedding layer, which initializes user/item embeddings and generates the rating-observed information representations for users.2) A preference encoder, which contains a preference aggregator to achieve NP of the support and query set, and a preference remainer to enhance common preference representations from the support set.An adaptive conditional decoder to incorporate features from the preference aggregator and preference remainer, and finally predict ratings of items in the target domain for the cold-start users.

Characteristic Embedding Layer
In CDR, our goal is to predict    in the ( \  ,    ) pair for the coldstart users, given the (   ,    ) pairs of overlapping users.Here   (with superscript  \  or ) summarizes the rating-observed information   ≜   = (  , V  (  ),    ) for user   to choose    with the user's interactions V  (  ) in the source domain.To achieve the above goal, we initialize the user/item representations in both domains, and then propose an attentive characteristic preference module to adaptively capture user's interactions V  (  ) in the source domain.Finally, we fuse the user's information from the source domain with the candidate item    in the target domain, so as to complete the rating-observed information   .
Specifically, we first initialize user embeddings as   ∈ R  , and items embeddings as    ∈ R  2 in the source domain and    ∈ R  2 in the target domain 2 .Intuitively, various items that users interact in the source domain contribute differently to their preferences.Therefore, we propose an attentive characteristic preference module over these item embeddings with attention mechanism [36]: , where  ∈ R  and   ∈ R  × are learnable parameters. ′  is the attention score of item    , differentiating the contribution of item    to the user preference.Thereby,    ∈ R  2 summarizes the user-specific preference derived from the source domain.
To predict rating    of candidate item    for each user, we enhance the user feature (including   and    ) with the item embedding    in the target domain.Specifically, we first fuse   and    to generate the user feature derived from the source domain, and then associate it with the candidate item: where we resize    ∈ R  2 into R  × , which serves as a userspecific projection    in linear transformation, allowing better user feature fusion [40].Besides, [•||•] denotes concatenation operation.Finally, we output   ∈ R  as the rating-observed information representation for the user   with candidate item    .Note that in the training phase, we leverage overlapping users in both Q  and C  , so we can learn how to make prediction on Q  by imitating C  .In the testing phase, we make predictions for cold-start users in Q  , by imitating the overlapping users in C  .

Preference Encoder
This section introduces two modules: 1) a preference aggregator to capture preference correlations among users via NP; 2) a preference remainer to enhance original user preference features.As suggested in previous NP studies [8], we incorporate the (  ,    ) pair to approximate the stochastic process with neural networks, which can be formulated as:

Preference
where   ∈ R  , and    ∈ R 1 denotes a vector with one dimension of the scalar    .Here we use MLP(•;  ) with three-layer neural networks parameterized by  to fuse the features of   and    .Afterward, to obtain task representation r , we aggregate  , with a commutative operation as in NP studies [8].Taking (  |C  ) as an example, the aggregation process is as follows: where ⊕ denotes a commutative operation, achieved by vector addition operation in our model for computational efficiency.Note that r is a permutation-invariance representation, which is a crucial property to ensure the ℎ condition of stochastic process [24].Such a design reduces the time complexity to  ( + ), where  and  are the number of users in C  and Q  , respectively.
To derive the random variable   of the stochastic process, as widely studied in variantional frameworks [12], we employ Gaussian distribution to represent   , i.e., imposing   ∼ N (  ,  2  ).Therefore, we generate   from the task representation r as: r = ReLU(  r ), where   ∈ R  × ,   ∈ R  × ,   ∈ R  × are learnable parameters.However, the random variable is intractable in training neural networks, we employ the well-known reparameterization trick [12] to sample data from probability distribution, thus the gradients can be smoothly backpropagated: where ⊙ denotes the element-wise product operation,  is the Gaussian noise.In this way, we can sample   from both (  |Q  ) and (  |C  ) in Q  and C  , respectively.Besides, by formulating all tasks involving users' interactions into the same stochastic process, we capture preference correlations among users for the CDR problem.

Preference Remainer.
To further enhance the preference correlations among users in different layer-depths, we devise a preference remainer to retain the original overlapping users preference features from the support set C  .Specifically, we first generates user preference representation  , by fusing rating-observed information    and the item rating    in the target domain, observed in the given support set C  as: where MLP(•; ) denotes three-layer neural networks parameterized by  and [•||•] denotes the concatenation operation.Then, we derive the common preference representation   by aggregating each user preference representation  , as follows: Here we use the same operation in Eq. (10) for convenience.In this way, we derive the common preference representations from the support set C  , enriching the preference correlations among users for making prediction in the query set Q  .

Adaptive Conditional Decoder
To make prediction for rating    given   in Q  , we have obtained the preference information from C  , including   from the preference aggregator and   from the preference remainer.To better incorporate   and   for predicting item ratings, we propose an adaptive conditional decoder based on the well-designed FiLM [5,25], which can fully modulate the two features in different layer-depths for decoding.
Specifically, the adaptive conditional decoder learns the conditional likelihood  (   |  ,   ) (in Eq.( 5)) with the   and   (in Eq. ( 12)) as inputs.We first generate two modulation coefficients   and   over the representation   , and then the adaptive conditional decoder can be formulated as: where   is the input of -th layer and [•||•] denotes the concatenation operation.We use tanh to enforce the modulation coefficients in range [-1,1].In this way,    controls the gain from   , while   remains the features of   .By stacking multiple layers, the adaptive conditional decoder fully captures preference correlations among overlapping and cold-start users in different layer-depths, and finally output the rating ŷ  of user   on item    at the last layer.

Loss Function
To train our model, the loss function consists of two terms: the desired log-likelihood to achieve CDR, and the KL divergence to impose regularization.Particularly, we rephrase the likelihood L  in Eq. ( 6) as a regression-based loss function by MSE loss: where we use overlapping users   1:|Q  | to replace  1:|Q  | .Besides, we impose L  for the  condition in the approximated stochastic process, which practically minimizes the KL divergence in Eq. ( 6) between the approximate posterior of the query set Q  and conditional prior of the support set C  : In summary, the overall loss function is defined as follows: where  ∈ [0, 1] is the hyper-parameter to balance losses.Note that we use overlapping users in the training phase, CDRNP first learns on support set C  and then updates according to the prediction loss over query set Q  .In the testing phase, we randomly removed all the ground-truth rating    in Q  and regard them as the test users (cold-start users).CDRNP can predict the preference of the cold-start users in Q  via a new task   with only overlapping users in C  available and the global information learned from the training phase.Thus given Q  and C  , the decoder takes   sampled from (  |C  ) and  \  in Q  as inputs to predict ŷ  for Q  .

EXPERIMENTS
In this section, we compare our proposed model with previous models, and conduct experiments to evaluate our model.

Datasets
Following existing methods [10,37,39] on CDR, we evaluate CDRNP on a real-world recommendation dataset, namely the Amazon review dataset.Specifically, we use the Amazon-5scores dataset where each user or item has at least five ratings.Following [10,37], we select three popular categories out of 24 categories for CDRNP: movies_and_tv (Movie), cds_and_vinyl (Music), and books (Book).We define three CDR scenarios: Movie-domain→Music-domain, Book-domain→Movie-domain, and Book-domain→Music-domain.While many existing works only evaluate a subset of data, we conduct experiments using all available data, which more realistically simulates the real-world.Table 1 summarizes the detailed information of three CDR scenarios.

Evaluation Metrics.
Following previous work [20,37], we use mean absolute error (MAE) and root mean square error (RMSE) as evaluation metrics to assess the recommendation performance.For each metric, we report the average results over five random runs for all users.Following previous meta-learning recommenders [1,3,13], we only predict the score of each user on candidate items in the query set Q  .

Compared
Baselines.For a fair comparison, we choose the following methods as baselines: (1) TGT represents the target MF model, which only uses data from the target domain for training and ignores user interactions in the source domain.( 2) CMF [31] leverages the source domain interactions as additional features for overlapping users to achieve performance improvement in the target domain.(3) EMCDR [20] follows the embedding and mapping idea for CDR, learning specific user representations in each domain through a mapping function, which is then utilized to transfer user representations from the source domain to the target domain.( 4) DCDCSR [38] is a CDR model based on matrix factorization and DNNs.It generates latent vectors for users and items using MF model, then uses DNNs to map latent vectors across domains.( 5) SSCDR [10] proposes a CDR framework based on semi-supervised mapping.It trains the mapping function by exploiting overlapping users and all items.Moreover, it predicts the latent vectors of cold-start users by aggregating neighborhood information.( 6) PTUPCDR [40] is a meta-learning-based CDR method that generates personalized mapping functions for each user by learning a meta-network, transferring the user's preference from the source domain to the target domain.

Implementation and Hyperparameter
Setting.The source codes for all baselines have been released, and we modify the data input and evaluation sections to fit our experimental settings.To ensure a fair comparison, we set the same hyper-parameter settings for all methods.For each method, we tune the dimension of embedding in {32, 64, 128, 256}.All models are trained for 10 epochs to achieve convergence.We use the same fully connected layer to facilitate comparison for CDRNP and other baselines.The stacking layers for preference aggregator, preference remainer and adaptive conditional decoder in our model are three, for attentive characteristic preference module is two.The learning rate is chosen from {0.001, 0.005, 0.01, 0.02, 0.1}.The hidden layer size for each layer is chosen from {8, 16, 32, 64, 128}, and the hyper-parameter  in Eq. ( 18) is chosen from {0.1, 0.3, 0.5, 0.7, 0.9}.In our experiments, we set the proportion of the testing dataset users  to be 20%, 50%, and 80% of the total overlapping users, respectively.We consider the sequential timestamps to avoid information leakage.

Performance Comparisons on CDR
This section presents the experimental results and detailed analyses of CDRNP in three CDR scenarios and in-depth discussion of the CDRNP bi-directional recommendation performance for cold-start users in both the source and target domains.

Performance Comparisons.
Table 2 shows the experimental performance of all methods in three CDR scenarios.Note that the size of C  is set to 40, and the length of interaction items history V  (  ) is set to 20.Based on the experimental results, we can draw the following conclusions: (1) CDRNP consistently exhibits the best performance in all CDR scenarios.Specifically, under the experimental setting of  value at 20%, CDRNP outperforms the existing SOTA (i.e., PTUPCDR) in MAE by 30.68%, 11.27% and 39.34% in three CDR scenarios.(2) CDRNP demonstrates outstanding performance and robustness even when the training dataset is limited.For example, for Movie domain→Music domain, with  value set to 20%, 50%, and 80%, CDRNP achieves improvements of 30.68%, 37.76%, and 41.06%, respectively, compared to the state-of-the-art meta-learning method.This indicates that our NP framework can effectively adapt to CDR even with limited trainable overlapping users.(3) CDRNP achieves competitive performance compared with all EMCDR-based and meta-learning-based baselines.The results highlight the necessity of capturing the preference correlations among overlapping and cold-start users.

Bi-directional Recommendation
Performance.We evaluate the bi-directional recommendation performance of CDRNP to coldstart users in both the source and target domain with EMCDR and PTUPCDR.Specifically, we first split 50% of the overlapping users for training EMCDR and PTUPCDR in the Movie-domain→Musicdomain.To ensure fairness, we split 25% of overlapping users for testing EMCDR and PTUPCDR in Movie-domain→Music-domain and the remained 25% of overlapping users for testing in Music-domain→Movie-domain, respectively.We take the same operation to train CDRNP, EMCDR and PTUPCDR in the Music-domain→Moviedomain, where testing them in the Music-domain→Movie-domain and Movie-domain→Music-domain. The empirical results are presented in Figure 3, from which we can draw the following conclusions: (1) CDRNP achieved the best bi-directional recommendation performance to cold-start users in both the source and target domains.

Ablation Study
In this section, we conduct an ablation study to test the effectiveness of each module in CDRNP.We first remove the Preference Remainer (PRM) module (FiLM layer number 0) from CDRNP, and then remove the Attentive Characteristic Preference (ACP) module based on that.We evaluate the intact CDRNP (FiLM layer number 3).We utilize the  value of 20% for all CDR scenarios.From the results shown in Table 4, we have several findings: (1) The CDR performance of CDRNP(w/o PRM) significantly outperforms that of CDRNP(w/o PRM+ACP).The results indicate the effectiveness of transferring the preference of users from the source domain to the target domain, and highlight the necessity of using the ACP module to utilize auxiliary information from the source domain for CDR.
(2) The improvement of CDRNP over CDRNP (w/o PRM) is significant.In this section, we modify the length of interaction items history V  (  ) and the number of users in C  separately in three CDR scenarios with  value of 20% to test the effectiveness of CDRNP.4.5.1 Impact of |V  (  )|.In this section, |V  (  )| is chosen from {10, 30, 50, 70, 90}.From the results shown in Figure 4, we can draw the following conclusions: (1) Despite a short interaction items history, CDRNP exhibits exceptional recommendation performance.Our model shows robust performance with regard to the length of interaction items history in the source domain.(2) CDRNP consistently outperforms CDRNP(w/o PRM) and PTUPCDR when the interaction items history is reduced.This indicates that our model is more effective in generating accurate recommendations using limited interaction items history in the source domain.(3) Our model is less affected by the length of the interaction items history.It is possibly due to its ability to model preference correlations among the overlapping and cold-start users and user-specific preference from both domains, resulting in complete preferences of users.In contrast, PTUPCDR is more sensitive to the change of |V  (  )|.

Hyper-parameter Analysis
In this section, we investigate the sensitivity of CDRNP to the hyperparameter  in Eq. ( 18).This experiment is conducted in three CDR scenarios with  value of 20%. Figure 6(a) shows results of CDRNP, and Figure 6(b) for CDRNP(w/o PRM).From these results, we can draw the following conclusions: (1) We find  can be set as a larger value for larger interaction scenario, such as the Book-domain→Movie-domain.The reason might be that the larger interaction scenario contains more complex preference correlations among the overlapping and cold-start users, which should be captured using a larger .In contrast, for the other two scenarios, a smaller  can achieve good recommendation performance.(2) The experimental results of our model are stable, which verifies the robustness of our model to changes in .Even in the worst case of , our model still outperforms other baselines in Table 2.

RELATED WORKS
Cross-domain recommendation leverages observed interactions from source domian to improve the recommendation performance in the target domian.the collaborative filtering (CF) paradigm, a significant amount of CDR methods [9,26,31,35] have been proposed to make effective recommender systems.Although these CF-based methods achieve outstanding performance, they still suffer from the cold-start and data sparsity issues.As our CDRNP aims to solve the previous issue, we only focus on CDR methods which recommend items to cold-start users.
To mitigate the cold-start issue, several CDR methods have been proposed to establish correlations between the source and target domains.Specifically, EMCDR [20] follows the embedding and mapping paradigm, which learns user representations in each domain with a mapping function.DCDCSR [38] utilizes MF and DNN to generate benchmark factors from source and target domains.SSCDR [10] prposes a semi-supervised mapping function by exploiting overlapping users and all items.PTUPCDR [40] and TMCDR [39] follow the MAML [6] framework to learn a meta network to substitute the mapping function.CATN [37] proposes a model based on user review documents to learn the correlation between the source and target domains.CDRIB [2] attempts to capture the domainshared information via information bottleneck [32].Compared with them, CDRNP has important design differences that we capture the preference correlations among overlapping and cold-start users and utilize explicit user-item interactions from both domains.
Meta-learning is a learning-to-learn paradigm, aiming to improve the performance of new tasks through training on a distribution of related tasks.Inspired by this, meta-learning methods have been introduced to alleviate the cold-start problem in recommender systems, which include optimization-based methods [3,13,19], gradient-based methods [6,23,29] and model-based methods [7,8,28].MeLU [13] fit a candidate selection strategy into a standard model-agnostic meta-learning framework to determine distinct items for customized preference estimation.MAML [6] aims at learning model's initial parameters which are capable of adapting to new tasks with only a few gradient steps.The proposed CDRNP falls into the model-based methods, which utilizes NP [8] to capture the preference correlations among users.
Neural Process is a well-known member of stochastic processes family.On the one hand, NP can model the distribution over functions like Gaussian processes.On the other hand, NP is structured as neural networks which can generate predictions in a efficient way.NP are originally designed to the tasks of few-shot image completion [7,8], image recognition [34].Since the first model CNP [7], there have been numerous subsequent works to improve NP in various aspects.ANP [11] introduces a self-attention mechanism to improve the midel's reconstruction quality.BNP [14] uses bootstrap to learn the stochasticity in NP without assuming a particular form.Recently, a series of methods [4,16,17] utilizing NP have emerged in the field of recommender systems.TANP utilizes Neural Process to solve the cold-start issue.However, TANP focuses on single-domain recommendation, which has quite different task setting comparing to cross-domain recommendation.

CONCLUSION
In this paper, we propose a novel meta-learning-based method CDRNP for CDR to cold-start users via Neural Process.CDRNP captures the preference correlations among overlapping and cold-start users with an associated stochastic process.Additionally, CDRNP introduces a preference remainer for enhancing the common preference from the overlapping users, and an adaptive conditional decoder to make prediction.Experimental results demonstrate that CDRNP consistently outperforms SOTA methods in three CDR scenarios.In the future, we will further investigate Neural Process for multi-domain recommendation.

Figure 1 :
Figure 1: Users 1 and 2 are overlapping users, while User 3 is a cold-start user.(a) The framework of EMCDR-based methods by learning a user-shared mapping function.(b) The framework of meta-learning-based CDR methods by learning user-specific mapping functions.(c) The framework of our CDRNP by capturing preference correlations among users.

Figure 2 :
Figure 2: An illustration of our model CDRNP with NP in the training and testing phase.

Figure 3 :
Figure 3: The framework of CDRNP in the training phase.The characteristic embedding layer is used to generate    .Both of the C  and Q  are encoded to generate the variational prior and posterior.  learned from the preference remainer is used to modulate the decoder parameters.  sampled from (  |Q  ) is concatenated with    in Q  to predict ŷ  for Q  via decoder.

Figure 4 :
Figure 4: Performance comparison of CDR to cold-start users with different lengths of interaction items history V  (  ).

Figure 5 :
Figure 5: Performance comparison of CDR for cold-start users with different lengths of support set C  .The results demonstrate the importance of capturing the preference correlations among users in different layer-depths.

Figure 6 :
Figure 6: Effect of the training loss factor .
for users: For each user   ∈ U, to predict rating  (  ,    ) in the target domain, we briefly summarize the rating-observed information as   = (  , V  (  ),    ).Here we use   ≜   and    ≜  (  ,    ) for brevity.Based upon, we define the concept of task   in meta-learning, where each   = {(  ,    )|(  ,    ) ∈ {C  ∪ Q  }} batched with multiple users.Here, we divide   into the support set C  and the query set Q  , such that   = C  ∪ Q  .Note that we train our model on the tasks   ∈ Ω − consisting of the overlapping users, i.e., we use (   ,    ) pairs with    ∈ U  in both C  and Q  , so as to learn the common knowledge of achieving CDR with these tasks.In the testing phase, our model predicts the interactions of the cold-start users in target domain via new tasks   ∈ Ω − , where we use (   ,    ) pairs with    ∈ U  in C  and we predict ( to learn it.Let (  |  ) be the variational posterior of the latent variable   , the corresponding evidence lower-bound (ELBO) derivation is as follows: Aggregator.Recall that, we regard each task   = {(  ,    )|(  ,    ) ∈ {C  ∪Q  }} decoupled from the same stochastic process   , and we approximate   by representation   generated from neural networks.Therefore, given Q  and C  , we aim to generate the variational approximations (  |Q  ) and (  |C  ).

Table 1 :
Statistics of Three CDR scenarios (Overlap denotes the number of overlapping users).

Table 2 :
Performance comparison of CDR to cold-start users in three CDR scenarios.

Table 3 :
Performance of bi-directional recommendation.

Table 4 :
Ablation Study on three CDR scenarios.
4.5.2Impact of |C  |.In this section, we set |C  | to {20, 30, 40, 50, 60}.From the results shown in Figure5, we can draw the following conclusions: (1) Both CDRNP and CDRNP (w/o PRM) maintain robust performance when the number of users in the support set C  is reduced.This indicates that our model is less affected by the reduction of users in C  .The stability may be attributed to the fact that CDRNP applies a random function to the constructions of   to simulate different function realisations, making the performance of CDRNP more stable.(2) CDRNP outperforms CDRNP (w/o PRM) in all experimental settings.The results further emphasize the importance of capturing preference correlations among the overlapping and cold-start users in different layer-depths.