Personalized Recommendation Algorithm for Mobile Based on Federated Matrix Factorization

There is a problem that the amount of users’ preference data on the mobile is small, and users are unwilling to disclose the preference data for the recommendation system about mobile users, so the server can’t centrally train a large amount of users’ preference data for a personalized recommendation. This paper proposes a personalized federated matrix factorization algorithm by introducing a federated matrix factorization model. The algorithm introduces users’ and items’ biases to modify the predictive rating model on each mobile; At the same time, conformity is introduced to give different weights to the preference data. In the case that the preference data does not leave the mobile, but the user preference data of the multiparty mobiles is shared, the multi-party mobiles and the server jointly train personalized matrix factorization model. The experimental results show that the algorithm in this paper still has high recommendation accuracy under the premise of correct updating in the federated matrix factorization model that uses bias and conformity.


Introduction
With the rapid development of mobile Internet and communication technology, people have got rid of the shackles of fixed terminals. By carrying portable smartphones, wearable devices, tablet computers, and other mobile devices, People can conveniently work, communication, social interaction, shopping, and other activities on the Internet anytime and anywhere. Data on activities is collected by mobile devices in real-time and regularly sent to third parties, such as service providers or institutions, then using data mining for market analysis or user recommendation. As a classic application of the machine learning model, the user recommendation system for mobile devices effectively solves the problem of information overload through a personalized recommendation of users by a third party. Through mobile devices for in-depth mining, this recommendation system collects a large amount of historical behavior data of users and analyzes the preferences and needs of different users by using mathematical tools, which can achieve accurate push to users. Such as product recommendation, advertising, etc. But at the same time, it may also disclose users' privacy, such as user's personal information, health status, geographical location, etc.

Federated matrix factorization
The goal of the algorithm in the recommendation system is to explore the connections between users and content products. Yang et al. [16] summarized federated recommendation algorithms into three categories, including horizontal federated recommendation algorithm (product-based federated recommendation), vertical federated recommendation algorithm (user-based federated recommendation), and migration federated recommendation. Horizontal federal recommendation mainly solves the problem of how to build collaboratively a recommendation system when participants have a large number of the same products and different user groups. For example, the federation of data between branches of the same movie company in different regions. Vertical federal recommendation mainly solves the problem of how to build collaboratively a recommendation system when the participants (institutions) have a large number of the same users and different products. For example, a federation of news recommendation services providers and video recommendation service providers, or a federation between recommendations service providers and user data providers. The migration federation recommendation mainly solves the problem of how to share collaborative experience to build a recommendation system when the participants have not many users and products. This paper focuses on the application scene in Figure 1 (different mobile users but the same products). For example, the products (movies) faced by users on a certain movie website are the same, but mobile users are different. Therefore, this paper mainly uses the horizontal federated matrix factorization recommendation algorithm as the basis to modify the recommendation accuracy. The concepts of additive homomorphic encryption, horizontal federated matrix factorization scene, and algorithm processes involved are as follows.
2.1.1. Additive homomorphic encryption. Additive homomorphic encryption (HE) is an encryption method commonly used in federated matrix factorization to protect user privacy through the exchange of encryption parameters. Homomorphic encryption allows any third party to operate the encrypted data without prior decryption. This paper uses a classic additive encryption scheme Paillier [22]: The additive homomorphic encryption has the following properties: (1) Encrypted numbers can be multiplied by non-encrypted scalars; (2) Encrypted numbers can be added together; (3) Encrypted numbers can be added to non-encrypted scalars. The encryption formula is generally as follows: Where E is the encryption algorithm and X is the collection of all data. It usually consists of the following functions: key generation, encryption process, decryption process, ciphertext addition operation, decryption, to get the added ciphertext.
2.1.2. Horizontal federation matrix factorization scene. The horizontal federated matrix factorization algorithm can be divided into two scenes according to the different rating data parties [23]. The first scene is that the rating data party is a mobile user of the recommendation system. The second scene is that the rating data party is a data party with a large amount of user data (such as movie websites, ecommerce websites). The specific scenes are as follows: (1) The data party is the mobile user of the recommendation system As shown in Figure 1, taking three mobile users as an example, the mobile users of the recommendation system have their rating data. And the user is a node, each user saves his row in the rating matrix locally. This row includes the user's ratings for different items, and when the data is not available on the mobile, its rating data is trained in the federated recommendation model in collaboration with other users.
(2) The data party is an institution with a large amount of user data As shown in Figure 2, two institutions as an example, the rating data owned by different institutions are about the same item, but the user groups of the two institutions are different. each data party has data in all the columns of several rows of the rating matrix, these rows correspond to the user groups owned by different data parties. For example, user rating data from different regions of the same movie company. In this case, each data party has that the rating data does not come out of the local data party, these data parties can still use their data to collaboratively train the federated recommendation model.  The loss function of the FedMF algorithm is based on the loss function of the matrix factorization, which has been widely used to mine the commodities. Funk [17] pointed out that the rating dataset of m users on n items in the original data can be transformed into a user-item rating matrix r. The original rating matrix r is decomposed into a user matrix p and an it-em matrix q with dimensions mathematically expressed as r p q   attributes of the item, each row in q represents the weight of the attribute or feature The predicted rating ,i j r of user i for item j can be obtained by the formula and . j k q denote the i-th row of the potential factor matrix p and the j update of user matrix q and item matrix q, this p [15]. The measurement standard for the two is determined by the minimized loss function As shown in Figure 2, two institutions as an example, the rating data owned by different institutions are about the same item, but the user groups of the two institutions are different. each data y has data in all the columns of several rows of the rating matrix, these rows correspond to the user groups owned by different data parties. For example, user rating data from different regions of the same movie company. In this case, each data party has a large number of users' rating data. In the case that the rating data does not come out of the local data party, these data parties can still use their data to collaboratively train the federated recommendation model. Diagram of an institution with a large amount of user data The loss function of the FedMF algorithm is based on the loss function of the matrix factorization, which has been widely used to mine the interaction between users and [17] pointed out that the rating dataset of m users on n items in the original data item rating matrix r. The original rating matrix r is decomposed into a em matrix q with dimensions m k  and n k  , in which the hidden feature k is T r p q   . Each row in p indicates how much each user likes the different ach row in q represents the weight of the attribute or feature of user i for item j can be obtained by the formula , th row of the potential factor matrix p and the j-th row of q respectively. For the update of user matrix q and item matrix q, this paper adopts the stochastic gradient descent method [15]. The measurement standard for the two is determined by the minimized loss function As shown in Figure 2, two institutions as an example, the rating data owned by different institutions are about the same item, but the user groups of the two institutions are different. each data y has data in all the columns of several rows of the rating matrix, these rows correspond to the user groups owned by different data parties. For example, user rating data from different regions of the a large number of users' rating data. In the case that the rating data does not come out of the local data party, these data parties can still use their data of an institution with a large amount of user data.
The loss function of the FedMF algorithm is based on the loss function of interaction between users and [17] pointed out that the rating dataset of m users on n items in the original data item rating matrix r. The original rating matrix r is decomposed into a , in which the hidden feature k is . Each row in p indicates how much each user likes the different ach row in q represents the weight of the attribute or feature in the item [18]. , , , i k p th row of q respectively. For the aper adopts the stochastic gradient descent method [15]. The measurement standard for the two is determined by the minimized loss function 2 , i j e . (2) ) is a regularization term to prevent the model from overfitting.
This paper mainly focuses on the horizontal federated matrix factorization of a scene (1). As shown in Figure 1, assuming that each participant is a mobile user, each participant's data is the mobile user's historical profile or historical information (user's rating data on items), assuming that the federal recommendation system has x mobile users as 1 2 , , , x S S S  . The 1st to the x-th mobile user has 1 2 , , , x m m m  user groups respectively (the number of users in each user group is 1), and To protect the privacy of participants, the FedMF algorithm isolates user rating data to prevent the server from directly accessing user rating data. Minimize the loss function on the mobile and the server to update the user matrix p and the item matrix q to minimize the difference between the predicted rating ,i j r and the real rating , i j r on the mobile user. The matrices p and q can be determined by minimizing the loss function (3). Then the loss function of the federated matrix factorization is as formula (3): Where x S represents the historical dataset of the i-th user, h p represents the user matrix of the h-th mobile. , represents the regularization item of the item matrix q, and the hth mobile user matrix to prevent overfitting. The update of the user matrix and the item matrix uses the federated matrix factorization algorithm-FedMF [6] for training. The general model framework is shown in Figure 3. Assuming that the user and the server have realized the generation and distribution of the key, the server has the public key pk , the user has the same private key sk , the public key pk is shared by the server and all the users, and the private key sk is only on the user the specific training process is as follows: (1) Initialize the parameters. Including the user matrix p on the client and the item matrix q on the server, the server encrypts q with the public key pk , get the encrypted item matrix where PEnc is the encryption process of Algorithm Paillier ; (2) The server provides an encrypted q C matrix for all clients to download; (3) ⅰ. The user downloads q C from the server, decrypts it with the private key sk to obtain , , , t represents the number of iterations,  represents the learning rate, and j q represents the j-th row of the item matrix, h D represents the batch training data from the h-th client; ⅱ. The client derives q to obtain the gradient (1 2 ) (2), (3), (4) until the loss function (3) converges, and achieve the entire training process.   In the federated matrix factorization recommendation, how to fully mine the user's interest preference from the user's rating of the item is a key issue of the algorithm model. In actual scenes, some users have stricter ratings of items and higher requirements for the quality of products, and the ratings are lower than other more forgiving users. Such individual differences in user rating standards seriously affect the accuracy of the matrix factorization training model to predict ratings. Paterek et al. [1 added user and item deviations in matrix factorization to more accurately mine the interaction between users and items. Koren et al. proposed BiasMF [10], which improved the performance of the basic matrix factorization model by introducing deviation (bias) terms between users and items. According to the above literature, this paper proposes a formula for adding user bias and item bias to correct the forecast rating in the matrix factorization process. sent by each client, update the encryption item matrix . Repeat steps (2), (3), (4) until the loss function (3) converges, and achieve In the federated matrix factorization recommendation, how to fully mine the user's interest preference from the user's rating of the item is a key issue of the algorithm model. In actual scenes, some users ents for the quality of products, and the ratings are lower than other more forgiving users. Such individual differences in user rating standards seriously affect the accuracy of the matrix factorization training model to predict ratings. Paterek et al. [19] added user and item deviations in matrix factorization to more accurately mine the interaction between users and items. Koren et al. proposed BiasMF [10], which improved the performance of the basic ias) terms between users and items. According to the above literature, this paper proposes a formula for adding user bias and item bias to correct the Where b i represents the bias of user i, and b j represents the bias of item j.

Rating conformity
High-quality rating data of user items can also help federated recommendation algorithms mine users' interest preferences and establish user interest models more accurately. Literature [18] believes that each user's preference for items generally satisfies a fixed range which is called a rating trend. Abnormal ratings that deviate from the rating trend will affect the accuracy of the predicted ratings. All rating data in the traditional federated matrix factorization model are treated equally, each rating has the same weight. If the user ratings deviating from his rating trend in certain specific environments (misoperation or emotional fluctuations, etc.), the rating continues to maintain the same weight as other ratings, it will reduce the accuracy of the federated matrix factorization model for user feature learning. Therefore, it is necessary to further mine user and item interaction data, improve the quality of rating data, and reduce the impact of abnormal points in rating data. This considers whether the mobile rating data in the federated matrix factorization model meets the user rating trend. This paper defines the degree of compliance with the user rating trend as conformity. According to the literature [18], this paper defines the user rating conformity , i j  to measure the degree of deviation of the i-th user's rating of the j-th item from the user's rating trend. The formula for conformity with user ratings in this paper is as follows: Where , i j r is the real rating, i  is the average rating of user i, and , i j  is the rating trend of user i.  is a minimum value. Here, the value  is set to 0.0001 to prevent the rating value and the average value from being equal, causing the denominator to be zero. max r is the highest rating value, to prevent too high conformity. , i j  has added a bias item of 1, to ensure that the lowest rating conformance is 1 to avoid the occurrence of 0. tanh is a normalized function, to prevent the conformity difference between the ratings from being too large, and aggravating the sparsity of the data.

Federated matrix factorization model fusion bias and conformity
The traditional federated matrix factorization algorithm applies the idea of federated learning to the traditional matrix factorization algorithm. Under the premise that the third-party server does not collect user preference data, it can still make accurate recommendations to target users. In the whole process, the user preference data never leaves the local area, and all preference data can be shared. It plays a role in protecting the privacy of user preferences. However, these algorithms use the traditional matrix factorization model without considering the bias and conformity of the rating data. In this section, the algorithm proposed in this paper will be described in detail. The implementation process of the algorithm is divided into two stages. In the first stage, the personalized correction parameters of bias and conformance are added to the traditional matrix factorization model, and a personalized matrix factorization (PER-MF) algorithm is proposed; In the second stage, the idea of federated learning is introduced into the personalized matrix factorization algorithm, and a personalized federated matrix factorization algorithm is proposed.

Personalized Matrix Factorization
Because the matrix factorization algorithm does not fully mine the user's interest preferences and the conformity of the rating data, it is not suitable for actual recommendation scenarios. In this section, 8 this paper adds the proposed personalized correction parameters to the loss function of the matrix factorization model based on the user's predicted rating and the conformity of the rating data to obtain the loss function of PER-MF algorithm.
The steps of the proposed PER-MF algorithm are as follows: (1) Calculate the average rating of each user called i  based on all user preference data; (2) According to the user's average rating, using formula (5) to obtain the conformity , i j  of each rating data; (3) User preference data is transformed into user-item rating matrix r; (4) Randomly initialize user matrix p, project matrix q, user bias b i , project bias b j (5) According to formula (4) modeling, the prediction rating of model training is obtained; (6) Calculating the error between the predicted rating ,i j r and the real rating , i j r according to the loss function (6); (7) Experiment with the stochastic gradient descent method to update the element values in p and q and the element values in b i and b j ; (8) Repeat steps (5) and (6) until the loss function converges. After the server collects user preference data and uses PER-MF algorithm to recommend products to users, there is a risk of leaking user preference data. How to share the preference data of multiple mobile terminals without the preference data leaving the mobile. At the same time, the PER-MF algorithm is used for recommendation by joint multi-party mobile terminals and servers, so the PER-FedMF algorithm is further proposed.

Personalized Federation Matrix Factorization
Based on the PER-MF algorithm, through the introduction of the idea of federated learning, when the third-party server does not collect user preference information, the mobile can make accurate product recommendations to users by using the PER-MF algorithm. Therefore, the PER-FedMF algorithm proposed in this paper is obtained. The data source of the algorithm is distributed, rather than stored on a third-party server. The data source of the algorithm is distributed, rather than stored on a third-party server. Interaction data (ratings) between users and items and user personal data are only available on mobile, while project features are stored and shared on a third-party server. The algorithm framework is proposed for the federated matrix factorization of fusion bias and conformity personalized parameters, which is suitable for most recommendation fields. After the introduction of federated learning, it is necessary to jointly train a personalized matrix factorization model on the mobiles of all parties and the server. Therefore, based on formula (6), the loss function of this algorithm is obtained through federated learning: Where 1  , 2  , 1  , 2  represent regularization parameters, b i represents the bias term of the user i whose rating is corrected on the h-th mobile, and b j represents the bias term of the global corrected rating. represents the regularization item of the user matrix and user bias of the h-th mobile. The regularization term is to prevent the model from overfitting. Compared with the traditional FedMF algorithm, how to update the personalized correction parameters of the user, item bias, and rating data conformity in the PER-FedMF algorithm proposed is one of the main contributions of this paper. The algorithm in this paper uses the traditional stochastic gradient descent method [15] to train p, q, b i , b j on each mobile and server. The generation and distribution of public and private keys are the same as the FedMF algorithm model. The framework of the algorithm in this paper is shown in Figure  4, and the pseudo-code is shown in Algorithm 1. The specific steps of the algorithm are as follows: (1) According to the modeling of the rating conformity in section 2.3, the rating dataset of the mobile is preprocessed to obtain the rating trend i  of user i and the conformity of the rating data , i j  .and update the dataset of each mobile as Ratings.
(2) Initialize the parameters, including the user bias b i on the mobile and the item bias b j on the server, The server uses the public key pk to encrypt b j and obtains the encrypted project offset

7: end while
According to the training process of PER-FedMF algorithm, during the execution of the algorithm, only the model parameters are alternately updated between the mobiles and the server. The user's original rating information does not leave the mobile, but the rating information can still be shared, which not only plays a role in protecting user privacy but also makes accurate recommendations. Compared with the FedMF algorithm, the algorithm in this paper considers and proves that the user, item bias and rating conformity parameters are correctly encrypted, transmitted, and updated between the mobile and the server, and it improves the accuracy of the recommendation. The rating conformity parameter is always transmitted throughout the model training process along with the square of the difference between the real rating and the predicted rating in the loss function (7); The user bias is only updated on the local mobile, the gradient of the item bias is updated on the mobile, and the update is encrypted and sent to the server, and the item bias parameters are updated on the server.

Experiment and analysis
This section will describe the datasets, experimental parameter settings, measurement indicators, evaluation methods, experimental results, and analysis used in the experiments in some detail in this paper.

Dataset
In this experiment, we select a representative MovieLens real movie rating dataset [21], which contains 100,000 rating information of 9724 movies produced by 610 users. Each rating ranges from 1   To explore the influence of the personalized parameters of bias and conformity on the results of federated recommendation, the recommendation quality of the algorithm in this paper is specifically measured in terms of the number of iterations, epochs, learning rate, and hidden features.
(1) Analysis of the impact of epochs on algorithm performance: It can be seen from Figure 6 that under the same number of iterations, PER-FedMF has a smaller RMSE value than FedMF, indicating that PER-FedMF has better performance than FedMF. Recommendation accuracy is highest when the number of iterations is small, because of the early stage of model training, user and item bias have a greater impact on the prediction rating. PER-FedMF approaches convergence faster than FedMF, and the convergence effect is better because the bias term is considered in the model training process while reducing the influence of abnormal preference data and further improving the prediction rating.
(2) Analysis of the impact of learning rate  on algorithm performance:  13 Here, the implicit feature factor k is set to 10, and other parameters remain unchanged. The range of learning rate  is 0.0001 to 0.001, and the RMSE value is used to judge the accuracy of the PER-FedMF algorithm. It can be seen from Figure 7 that under the same learning rate  , the RMSE value of PER-FedMF on the test set is smaller than that of FedMF, indicating that PER-FedMF has a betterrecommended performance. Especially when the learning rate  is small, the PER-FedMF recommendation accuracy improvement effect is more obvious. This is because when the learning rate  is small, the model has not yet reached convergence during the 150 iterations. The user, item bias, and conformity have a greater impact on model training. With the gradual increase of the learning rate between the value ranges, the performance improvement of the algorithm reaches a stable state relative to FedMF, because the degree of correction of the predicted rating becomes stable during the model convergence process of the personalized correction parameter. In the case of a certain number of iterations, a small learning rate may prevent the model from converging to the minimum value, and a large learning rate may cause the model to float back and forth near the minimum value, neither of which can reach the optimum. Therefore, the appropriate learning rate should be selected according to the actual situation.
(3) Analysis of the impact of implicit feature k on algorithm performance Here, the learning rate  is set to 0.0002, and other parameters remain unchanged. The range of hidden feature k is selected as 10-50, and the accuracy of PER-FedMF is judged by MAE. It can be seen from Figure 8 that under the same hidden feature k, the MAE value of PER-FedMF is smaller than that of FedMF, indicating that PER-FedMF has a better-recommended performance. When the hidden feature k is small, the performance improvement effect of PER-FedMF recommendation is more obvious, because the model does not fully mine the preference data at this time. At this time, adding users, item bias, and conformity can mine user preference data to a greater extent. As the value of K increases, the degree of performance improvement of PER-FedMF is gradually decreasing, because as the value of k increases, FedMF's mining of user preference data becomes more and more sufficient, and the degree of correction of the prediction rating by the personalized correction parameter is reduced, but within the selected k value range, the performance of the PER-FedMF algorithm is always optimal.
After the above-mentioned experimental results and analysis, this paper proves that the personalized correction parameters of the user, item bias, and conformity are correctly updated in the FedMF model. It is verified that the user preference data does not leave the mobile, and the PER-FedMF model with almost lossless recommendation accuracy can still be trained; Compared with the traditional FedMF algorithm, the recommendation performance of the PER-FedMF algorithm in this paper is better and is more suitable for real recommendation scene.

Conclusions
This paper proposes a mobile personalized recommendation algorithm PER-FedMF based on federated matrix factorization. The algorithm introduces the personalized correction parameters of the user, item bias, and rating conformity into the federated matrix factorization model, reconstructs the predictive rating model, and distinguishes the quality of rating data. Under the condition that the user rating data on the mobile does not leave the local area, the relationship between users and items is fully and accurately mined. Through experiments on the benchmark movie dataset, it is shown that PER-FedMF is better than the current FedMF model in recommendation accuracy and is more suitable for real recommendation scenarios. But in the process of PER-FedMF model training, the time efficiency is low. How to improve the time efficiency of PER-FedMF model will be a direction for our further research in the future.