With the rapid development of Location-based Social Network (LBSN) services, a large number of Point-Of-Interests (POIs) have been available, which consequently raises a great demand of building personalized POI recommender systems. A personalized POI recommender system can significantly assist users to find their preferred POIs and help POI owners to attract more customers. However, it is very challenging to develop a personalized POI recommender system because a user's checkin decision making process is very complex and could be influenced by many factors such as social network and geographical distance. In the literature, a variety of methods have been proposed to tackle this problem. Most of these methods model user's preference for POIs with integrated approaches and consider all candidate POIs as a whole space. However, by carefully examining a longitudinal real-world checkin data, we find that the whole space of users' checkins actually consists of two parts: social friend space and user interest space. The social friend space denotes the set of POI candidates that users' friends have checked-in before and the user interest space refers to the set of POI candidates that are similar to users' historical checkins, but are not visited by their friends yet. Along this line, we develop separate models for the both spaces to recommend POIs. Specifically, in social friend space, we assume users would repeat their friends' historical POIs due to the preference propagation through social networks, and propose a new Social Friend Probabilistic Matrix Factorization (SFPMF) model. In user interest space, we propose a new User Interest Probabilistic Matrix Factorization (UIPMF) model to capture the correlations between a new POI and one user's historical POIs. To evaluate the proposed models, we conduct extensive experiments with many state-of-the-art baseline methods and evaluation metrics on the real-world data set. The experimental results firmly demonstrate the effectiveness of our proposed models.