Help:Submitting A Ratings Assignment
The purpose of the the rotisserie rating system is to identify the most interesting threads within a single rotisserie. Previous rotisserie systems that we've built have lacked this feature, making the archives impossible to browse meaningfully by force of the amount of material generated. We have frequently hosted rotisserie discussions of several rounds with over two hundred participants, with each participant writing an average of a couple hundred words for each round. Simple math establishes that the volume of material generated by a single such rotisserie is too enormous for a single person without vast amounts of time to do anything other than read a few random threads. The result of this problem is that the rotisserie archives for large groups are largely useless, which is a shame because of the amount of interesting content buried in them.
We implemented the rating system specifically to combat the above problem. By allowing users to rate posts and then collecting these post ratings into average thread ratings, we are able to provide a rough guide to what are likely some of the most interesting threads within a rotisserie.
The key attributes of the rating system are:
- Each round, each user is required to rate at least 5 posts from the previous round (or all posts not written by himself if there are less than 5 posts in the previous round). This requirement guarantees that each post will be rated by at least 5 different users, providing a reasonable spectrum of opinions. A user is not required or allowed to rate his own post.
- A user must complete all rating assignments before responding within a given round.
- A user does not have to use up all of his rating points within a given round. Rating every assigned post '0' is acceptable.
- In addition to rating his assigned posts, any user participating in the rotisserie is allowed to rate any other post in the previous round via the various thread view pages. Rating points for rating non-assigned posts come from the same pool as those for assigned posts. So a user who wants to rate non-assigned posts should not use all of his points on the assigned posts.
- Each user is granted 1 rating point for each post she is assigned to rate within a given round. So if the user is required to rate 5 posts, she is given 5 points for the round.
- The rating points are valid only for the round in which they are granted. A user cannot save rating points and use them in a different round or a different rotisserie.
- Each rating point can be used to alter the rating of a given post by 1 point in either direction. So it costs 2 rating points to give a post either a +2 or a -2 rating.
- Individual post ratings are the sum of the ratings of all users. So if a post is given ratings of +2, +1, -1, +2 by 4 different users, the rating of that post is +4.
- Thread ratings are the average ratings of all posts within the thread. By averaging instead of summing the post ratings within a thread, we guarantee that thread ratings between different rotisseries have comparable significance (so a thread with a rating of +10 is always a fantastically interesting thread, while a thread with a rating of -3 is always mildly boring).
- Post ratings are only visible to project leaders. Since the purpose of the system is to identify interesting threads, there's no real need to display the ratings of individual posts, and we fear that allowing participants to view the ratings of individual posts will often result in a horse race competition atmosphere, which is not the point (sotospeak) of the system.
- A project leaders have the option of turning off the rating system for any rotisserie led by her.