Evaluate ALS

[sumedhvdatar]

I am trying to use Implict ALS in production. I have a dataset with implicit feedback and I need a way to evaluate my training loss and validation loss. I need to check if my model is overfitting or not.

Currently, the model after training is giving very bad recommendations and I need to fine tune it more. Need help here!

[ruojol]

What kind of mean average precisions are you getting? http://fastml.com/what-you-wanted-to-know-about-mean-average-precision/
f.ex.
from implicit import evaluation evaluation.mean_average_precision_at_k(model, USER_ITEM_train, USER_ITEM_test, k=k, show_progress=False, num_threads=0)

I also have little followup question for the developers on this.
Is it possible to output the training errors anyother way than through verbosity? Or do I have to impelement this myself to the package?
A workaround would be that one cloud calculate f.ex. the mAP @ k after every x epochs? Then you could monitor the model training through tensorboard with a little tensorflow summary writer magic.
I've implemented an implicit model in google ml engine and did some hyperparameter optimization with it but it would be great if you could output the training loss during training because now I have to train the model and then calculate mAP and compare the tuning results for maximum mAP. And like @sumedhvdatar I'm also at a little loss if the model is over or under fitting during this process.

[H0bbitBaron]

You can output at least mse while training like described here - #11.