Collaborative Filtering

User-Item Interaction Matrix ($R$)

Given $N$ users and $M$ items, interaction matrix $R \in \mathbb{R}^{N \times M}$ is extremely sparse ($> 99\%$ empty cells):

$$\text{Sparse Matrix } R_{N \times M} \approx U_{N \times k} \cdot V_{M \times k}^T$$

         Items (M)                                 Item Latent Factors Vᵀ [k × M]
     ┌──────────────┐                                   ┌──────────────────────┐
U  u │ 5  .  1  .  4│                              u    │ v1   v2   v3   ... vM│
s  s │ .  2  .  5  .│  ──Matrix Factorization──►   s    └──────────────────────┘
e  e │ 1  .  .  4  .│                              e  User Latent Factors U [N × k]
r  r └──────────────┘                              r    ┌──────────────────────┐
s (N)                                              s    │ u1   u2   u3   ... uN│
                                                        └──────────────────────┘

Predicted rating for User $i$ on Item $j$: $\hat{R}_{ij} = u_i \cdot v_j + \mu + b_i + b_j$.

Three Paradigms of Collaborative Filtering

1. User-Based Neighborhood CF: Find $K$ most similar users to User $i$ using Cosine or Pearson similarity over shared rated items. Average their ratings.
   • Problem: Does not scale well when $N_{\text{users}} \gg N_{\text{items}}$ (User profiles shift constantly).
2. Item-Based Neighborhood CF: Find $K$ items most similar to Item $j$ based on co-rating patterns across all users.
   • Advantage: Item-item relationships are stable over time, enabling pre-computed item similarity matrices (Amazon: "Customers who bought X also bought Y").
3. Model-Based Matrix Factorization (SVD / ALS): Decomposes $R$ into latent embedding vectors $u_i, v_j \in \mathbb{R}^k$ ($k \approx 32\text{--}256$).

Explicit vs Implicit Feedback

• Explicit Feedback: Star ratings (1 to 5), Likes/Dislikes. Clean signal, but extremely rare.
• Implicit Feedback: Clicks, video watch time, page views, purchases. Abundant, but noisy (no explicit negative ratings).

Implicit ALS Objective (Hu, Koren, Volinsky)

$$\min_{U, V} \sum_{i, j} c_{ij} \left( p_{ij} - u_i \cdot v_j \right)^2 + \lambda \left( \sum_i \|u_i\|^2 + \sum_j \|v_j\|^2 \right)$$

• $p_{ij} = 1$ if interaction exists, else $0$.
• $c_{ij} = 1 + \alpha r_{ij}$ (Confidence score scaling with interaction intensity $r_{ij}$).

Say this out loud

"Collaborative filtering predicts user preferences using historical crowd interaction logs without requiring item metadata. Matrix factorization decomposes sparse User-Item matrices into dense low-rank latent vectors u_i and v_j, predicting ratings via dot product u_i · v_j. For implicit feedback like clicks and watch time, we use Alternating Least Squares (ALS) with confidence weighting."

Follow-ups to expect

• Why is SVD unsuitable for sparse matrices directly? Standard linear algebra SVD requires full dense matrices. Imputing missing values with zeros corrupts predictions. Matrix Factorization algorithms (SVD++ / Funk SVD) optimize SVD parameters strictly over non-missing entries.
• How do Hybrid Recommenders improve upon pure CF? Hybrid recommenders combine CF latent vectors with content-based features (text/image embeddings, user demographics) in a unified deep learning model (e.g., Wide & Deep, DeepFM), solving cold-start issues while preserving CF collaborative signals.