Confidence Matrix values should be 0 or 1?

[b-carter]

It seems that the code expects a sparse (e.g. csr) input containing the confidences on each of the UI pairs. However, looking at the Hu et al. 2008 paper, it seems like there should still be non-zero confidence for items that are unobserved (r_ui = 0 in the paper's notation), such as from c_ui = 1 + alpha * r_ui. But, doing this would make the confidence matrix dense.

I see that in computing the updates for the latent factor matrices, there is a (C - I) term and a multiplication by P, which transform back to the sparse space, but then shouldn't the input matrix be dense to accommodate the subtraction? (Ideally it isn't, but I'm trying to understand the implementation as written).

Thanks a lot for the explanation!

[benfred]

Its maybe not the most clear - but the 'Cui' matrix being passed in does double duty here: It defines which users for each item that liked it (Pui in original paper) - as well as defines how much confidence we have that the user liked that item.

The Pui values are implicit - the library is assuming that non-zero items in the Cui matrix means that the user liked the item (Pui = 1). The negatives are left unset in this sparse matrix: the library will assume that means Pui = 0 and Cui = 1 for all these items. This lets us only do work on the sparse liked items by leveraging that YtCuY = YtY + Yt(Cu - I)Y

[giasemidis]

Thank very much for this great piece of work. I had the same question and looked carefully into the code. Looking into the user_linear_equation function in als.py module, line 340, A += (confidence - 1) * np.outer(factor, factor) indicates that the confidence for the entries where Rui = 0 (the raw item-user matrix) should be 1. But this is not the case as Cui is sparce. If Cui < 1, for Rui > 0 (say, because Rui has fractions and alpha ~ 1), then (confidence - 1) contributes negatively to A. Or in other words, if confidence = 0 (because Rui = 0) then A += - 1 * np.outer(factor, factor), i.e. terms with no explicit feedback contributing to A? does this make sense?

Similarly, in b += confidence * factor shouldn't be `b += (confidence + 1) * factor?

In the limit where confidence >> 1, the two implementations should be the same.

Could you please elaborate what I have misunderstood?

[liesb]

Is it possible that the item_users matrix that is provided to fit is actually Ciu * Piu, i.e.,

• C[i, u] if P[i, u] = 1, e.g 1 + \alpha * r[i, u] in the linear transformation case (cfr the Hu paper), meaning we explicitly have to add the 1 into the item_users matrix? I seem to remember that this wasn't the case in an earlier version of implicit, but I might be wrong.
• 0 else.

I have a further question. In the function user-linear_equation, we test for confidence > 0 - does this imply there could be strictly negative values in the confidence matrix? If so, how can we interpret this, and doesn't the sign change (confidence *= -1) not cause trouble in the next line (`A += (confidence -1) * np.outer(factor, factor))?

Thanks in advance!