Log in | Register
Forum > Site Discussion > Thread

The GWM rating system: what does it measure?

Aug 28, 2010 - permalink
I apologize for having posted some of these observations in connection with a comment thread for a photo. Obviously, this forum is the appropriate place for it. There are some important considerations concerning the rating system! This is just a reminder to the site staff and the members of what we already know intuitively! First, no ratings are statistically meaningful unless the sample size is large enough. Three viewers rating a photo is just not comparable to one hundred rating it! the former case would be valuable as an opinion, but not generalizable. Personally, I prefer a sample size of one hundred or more.

Second, viewers should be encouraged to rate pictures according to how they actually feel/think rather than how somebody else does or how they think somebody else does. It's ok to rate a photo low, if it's not a prank or a conscious effort to mess up the rating (which practice has been alluded to in many comment threads)! In fact, we need more votes particularly in the mid-range and even at the lower end in order to get a more realistic picture of people's reactions! It doesn't mean anything if we only rate photos we love; we've gotta rate some we're less enthusiastic about! That's why there's a preponderance of 8.8-8.9 ratings on this site right now! Nine is by far the most common rating, because if you feel strongly about a photo, you're going to rate it high, but not too high so as to be out of line with other raters or to call attention to yourself, because, after all, we do value each other's opinions to some extent! A fellow-member of the site has made an interesting observation about the rating system. He/she suggests it might be useful to look at the ratio of the number of viewers favoriting a picture to the number of people rating that photo. It would tell what percentage of people that took the time to rate a photo call it a favorite. Ideally, that percentage should be maybe between 40 and 70: if the ratio is too low, then it indicates very few who rated it consider it a favorite (a distortion), and if the ratio is too high, it means the only people who rated it were those who really liked it (an all too common situation). But when the ratio is in the middle range, it means some of the people who rated the photo really liked it while others didn't (more realistic)! You could, of course, have the fairly uncommon case where more people favorite the photo than rate it! 

Third, are we rating the model, the particular pose, the quality of the photo, or some combination thereof? High resolution photos or morphed photos influence ratings to some degree. Fourth, it has been my experience that people rate differently when thay are forced to make a comparison.  A certain model may at first glance have excellent biceps, . . . until her photo is placed side-by-side with that of another model with more developed ones. It may be that the ratings of one or both models would change! Such comparisons would certainly result in a truer and more meaningful rating. So maybe the site could do some breakout pairing-up or group comparisons complete with ratings and comments pertaining thereto!  Of course, something like that is already being done in some of the polls, but this idea would involve pictures as well as text!

To sum up, we site members could perhaps benefit from rating more photos that don't necessarily bowl us over, comparitive ratings scenarios, and maybe we could pay a little more attention to those ratings that come from larger samples of viewers! IMHO! (:^{D)
Sep 08, 2010 - permalink
I think I mostly agree with you (although I'm not sure what you really need a sample size of > 100 *for*), but on the subject of "rating more photos that don't necessarily bowl us over" ...

yes, but there are *so many photos* coming in to the site these days that I rarely bother to even click on the thumbnail of anything I think looks like less than an 8.  Which is probably why my current average rating is 8.65.

This is also related to why I asked for a "pictures I haven't rated yet" filter, but that idea didn't seem to pick up much steam.
Sep 08, 2010 - permalink
Rating pix on the main site is purely an option as like leaving comments-I must confess plenty of pix are over-rated but plenty are under-rated',as the old saying goes you cant please people all the time-just some of the time,as to gracillis 8.65 that figure is in fact an average what other members have rated your uploads for example if you have posted 10 pix all been voted once with all 10 votes your average would be 10
Thread locked by cgsweat.
« first < prev Page 1 of 1 next > last »