Background
We assign weights to tags members follow based on what they explicitly set, and adjustments based on implicit activity. In the weighted algorithm, we had been taking into account whether people followed tags, but not the weights.
This experiment tested whether putting these points back in had a positive effect.
We allowed this test to run for two weeks because there was no runaway winner.
Results
Here were the possibilities:
- incumbent
- tag_follow_points_maximum_spectrum
- tag_follow_points_medium_spectrum
- tag_follow_points_minimum_spectrum
| Scenario | Incumbent ranking | 2nd Place | Likely Winner |
|---|---|---|---|
| Creates a comment. | 2nd | incumbent | tag_follow_points_minimum_spectrum |
| Creates comments on at least 4 different days within a week. | 3rd | tag_follow_points_minimum_spectrum | tag_follow_points_maximum_spectrum |
| Views pages on at least 4 different days within a week. | 2nd | incumbent | tag_follow_points_maximum_spectrum |
| Views pages on at least 4 different hours within a day. | 1st | tag_follow_points_maximum_spectrum | incumbent |
| Views pages on at least 9 different days within 2 weeks. | 3rd | tag_follow_points_minimum_spectrum | tag_follow_points_maximum_spectrum |
| Views pages on at least 12 different hours within five days. | 1st | tag_follow_points_minimum_spectrum | incumbent |
The results were fairly mixed, and there is no clear winner, but tag_follow_points_maximum_spectrum won the most tests with three wins.
Conjecture
This could arguably be an opportunity to keep the incumbent, given that it is not abundantly clear that tag_follow_points_maximum_spectrum should be the winner. However, given that without tallying points, we are not really respecting the functionality we offer to users to have their points accounted for — tie should go to the test that allows for this. Among the challengers, "max spectrum" was the winner. This will also allow us to fiddle with the points that tags are allotted independently to adjust the score in a different test.
There is a lot of benefit to declaring tag_follow_points_maximum_spectrum the winner.
Next steps
We have a new test scheduled for review and some adjustments coming down the pike to allow more flexibility for how other Forems might adjust their own feed strategies.
Top comments (0)