PrivateRec: Differentially Private Model Training and Online Serving for Federated News Recommendation

Federated recommendation can potentially alleviate the privacy concerns in collecting sensitive and personal data for training personalized recommendation systems. However, it suffers from a low recommendation quality when a local serving is inapplicable due to the local resource limitation and the data privacy of querying clients is required in online serving. Furthermore, a theoretically private solution in both the training and serving of federated recommendation is essential but still lacking. Naively applying differential privacy (DP) to the two stages in federated recommendation would fail to achieve a satisfactory trade-off between privacy and utility due to the high-dimensional characteristics of model gradients and hidden representations. In this work, we propose a federated news recommendation method for achieving better utility in model training and online serving under a DP guarantee. We first clarify the DP definition over behavior data for each round in the pipeline of federated recommendation systems. Next, we propose a privacy-preserving online serving mechanism under this definition based on the idea of decomposing user embeddings with public basic vectors and perturbing the lower-dimensional combination coefficients. We apply a random behavior padding mechanism to reduce the required noise intensity for better utility. Besides, we design a federated recommendation model training method, which can generate effective and public basic vectors for serving while providing DP for training participants. We avoid the dimension-dependent noise for large models via label permutation and differentially private attention modules. Experiments on real-world news recommendation datasets validate that our method achieves superior utility under a DP guarantee in both training and serving of federated news recommendations.