Comments on Pragmatic Theory: Blending 101

Training a different blending function for each us...

2008-08-13T13:12:00.000-04:00

Training a different blending function for each user is unlikely to succeed for lack of sufficient training data. However, training different blending functions for subsets of users is discussed in The BellKor Solution to the Netflix Prize (section "Combining multiple results").

Would it be feasible to have different per-user bl...

2008-08-13T05:50:00.000-04:00

Would it be feasible to have different per-user blending functions? Maybe each algorithm that goes in the "blender" has a group of users on which it works best...

LOL

2008-07-29T17:24:00.000-04:00

LOL

http://en.wiktionary.org/wiki/101

2008-07-29T10:52:00.000-04:00

http://en.wiktionary.org/wiki/101

Blending 101... What does the 101 mean ?

2008-07-29T09:17:00.000-04:00

Blending 101... What does the 101 mean ?

The submission process works as follow:1) When a r...

2008-07-28T18:48:00.000-04:00

The submission process works as follow:

1) When a result file is submitted, the last submit time is immediately updated on the leaderboard and a confirmation email is sent to the team leader.

2) The server computes the score. It takes a few minutes.

3) If the new result is the best so far, the best score column of the leaderboard is updated with the new value. An email with the new score is sent to the team leader, whether or not it improves on the previous scores.

So the team leader receives an email with the submission score, good or bad. The leaderboard always displays the best submission score ever, together with the latest submission date/time.

Also note that you appear in the leaderboard only once you have beaten Cinematch once.

I'm sure that you can find more details on this on the www.netflixprize.com website. Look at the forum discussions if nothing else.

Do you get results after you submit a resulting da...

2008-07-28T14:19:00.000-04:00

Do you get results after you submit a resulting dataset?

Or if it is lower than your last RMSE on the Leaderboard you don't get the results?

We have tried several blending methods, but I won'...

2008-07-27T10:28:00.000-04:00

We have tried several blending methods, but I won't go into details here. We used cross-validation by splitting the probe set to investigate different approaches and select parameters. For official submissions, we don't split the probe set.

Thanks for taking the time to write down all this ...

2008-07-26T10:35:00.000-04:00

Thanks for taking the time to write down all this information. Could you care to expand a bit more on your blending method?

I mean the actual blending recipe. Do you use some kind o neural network or just linear regression? Also, do you really use cross-validation (like splitting the probe in half or more parts and try combinations on those)?

My small contribution to the points made in this post: my gap between probe and test rmse is about 0.0065 +/- 0.0001 (so I can confirm your numbers)