Recommendation Systems: Collaborative Filtering & Embeddings

Personalized recommendation systems predict the affinity score \hat{r}_{u, i} between a user u and an item i based on historical interaction signals (explicit ratings, clicks, dwell times, purchases).

Classical Collaborative Filtering Paradigms:

1. User-Based Collaborative Filtering: Identifies nearest-neighbor users who exhibit similar historical interaction patterns to user u and recommends items those neighbors enjoyed:

Sim(u, v) = \frac{\sum_{i} (r_{u, i} - \bar{r}_u)(r_{v, i} - \bar{r}_v)}{\sqrt{\sum_i (r_{u, i} - \bar{r}_u)²} \sqrt{\sum_i (r_{v, i} - \bar{r}_v)²}}

1. Item-Based Collaborative Filtering: Computes item-item similarity vectors based on co-occurrence in user histories (e.g., Amazon’s "Customers who bought X also bought Y"). Item similarities are far more static and computationally efficient to cache offline than dynamic user similarities.
2. Matrix Factorization (Singular Value Decomposition / ALS): Decomposes the massive, sparse user-item interaction matrix R \in \mathbb{R}^{|U| × |I|} into low-rank latent user factors P \in \mathbb{R}^{|U| × k} and item factors Q \in \mathbb{R}^{|I| × k} (where k \ll |I|, typically 64 - 256 dimensions):

\hat{R} ≈ P Q^T \implies \hat{r}_{u, i} = p_u \cdot q_i