Should the scoring system of level reviews be abolished on here?

helping builders & reviewers hone their skills

Moderators: QRS, Raymond, eTux, EssGee

Post Reply
User avatar
DJ Full
Master Chatter
Master Chatter
Posts: 6800
Joined: 02 Nov 2010 03:02

Re: Should the scoring system of level reviews be abolished on here?

Post by DJ Full » 18 Jun 2018 06:06

Phil wrote:
18 Jun 2018 02:29
I think we've had enough of the gassy rhetoric, and since the person who makes the final decision has already expressed his opinion, it's time to move on.
We spoke because bringing new data sometimes changes such opinion, and opening up to twitch and meeting the streamers provided such. So don't talk to us like if things were static, like if nothing we could say had any value, because this sounds like you gonna dismiss anything we say, no matter what it is and what logic is behind it, before you even hear it. It doesn't feel good, and I surely don't now.
Mytly wrote:
17 Jun 2018 22:06
DJ Full wrote:
17 Jun 2018 21:49
Mytly wrote:The whole point of the categories is to enable people to distinguish between the categories. If you eliminate that, what's the score even based on?
On how the game feels. It's enough
And what exactly prevents you or anyone else from using the current system to do just that?
This is what I recently do. But before that I misrated lots of levels because I believed the guidelines are correct. I actually consider changing the rating of many levels I (mis)rated and I also think it should be more flexible and user-dependent than to have to bother Michael every time (which I'm aware is priceless). If I didn't have to do that, I would update already.
MAKE WIDESHOT GREAT AGAIN

User avatar
Mr XY
Member
Member
Posts: 962
Joined: 30 Jun 2006 16:59
Location: Heidelberg, Germany

Re: Should the scoring system of level reviews be abolished on here?

Post by Mr XY » 18 Jun 2018 10:49

Playing, building, reviewing, Tomb Raider in general is supposed to be a fun Hobby. But some people are taking this to a whole new level. Maybe it really is time to move on when it is not fun anymore? Or at least take a few steps back and look at it from a bit further away. A break to get your head free might just be the right thing. :angel:

The review system of one Tomb Raider fan page is surely not the reason why people choose to leave the community. We are all getting older, find new interests or just don't have the time anymore. And the classic Tomb Raider games are getting less and less relevant. And a new review system won't change all that.
Image

Marksdad
Mod
Mod
Posts: 4839
Joined: 05 Oct 2001 16:38
Location: UK

Re: Should the scoring system of level reviews be abolished on here?

Post by Marksdad » 18 Jun 2018 11:55

For a website that's been going strong for the past 18 years,and still maintains a solid amount of levels released per month (as well as having the most comprehensive TR Custom Level Listing on the web) I don't feel anyone need worry about it meeting its downfall any time soon simply because one disgruntled reviewer is of the opinion that his way is possibly better :wink:

Nonetheless Ceamonks (assuming you're still reading this) why not try one other alternative? Set up a website of your own,featuring a compendium of your own reviews,and with your own rating system?
It needn't be especially comprehensive;but you have a solid collection of nigh on 700 levels played and reviewed already;so why not copy the text of a bunch of them,along with your own standard of ratings?
I'd be genuinely interested in seeing how it compares.

User avatar
Jorge22
Master Chatter
Master Chatter
Posts: 9439
Joined: 21 May 2003 03:26
Location: Portugal (Porto area)

Re: Should the scoring system of level reviews be abolished on here?

Post by Jorge22 » 18 Jun 2018 15:23

Ooooh the younger generations are all being scared away from this God forsaken place because the reviews contain ratings, which is a totally old thing nobody ever uses anymore! :freakout:

Or maybe they're just too busy drinking and smoking weed while they party and rap and date and chat on their cells or whatever pleases them the most.

There must have been a reason why I completely forgot about this thread even though Alzheimer's keeping me from recalling what it was...
:thinking2:
Image

User avatar
DJ Full
Master Chatter
Master Chatter
Posts: 6800
Joined: 02 Nov 2010 03:02

Re: Should the scoring system of level reviews be abolished on here?

Post by DJ Full » 18 Jun 2018 16:46

I can definitely agree that I care too much but I prefer to care too much than to care too little. Because in 90% of cases when I cared too little nothing bad happened, yet it were 10% of cases when I should have cared and those were the cases which ruined things.

Mytly, what I precisely mean about the system "preventing" from rating according to my feelings is that it still has those categories, it still has criterions. You are right that they don't prevent ignoring them but you still need to take time and experience to figure it out. Very often I was torn between rating according to those guidelines, and to my wish of skipping them, simply because I didn't know which is better. And I know people who have the same issue. I remember izzy noodles beginning her reviewing career - she's a streamer, she was unfamiliar with our world, she failed to "feel" our scale properly - and so she rated objects in Beyond the Scion to 7/10 because she thought it's "quite good". I remember how upset EssGee was about this, and I still think he was right. If there was just one slider between 0-10, she would give 9 or 10 for the entire game. If there was a slider between 1 and 0, she would give a 1. I don't understand why people don't see the damage which is currently done.
MAKE WIDESHOT GREAT AGAIN

User avatar
rufierto
Member
Member
Posts: 510
Joined: 02 Jul 2011 23:22
Location: Centralia, Missouri

Re: Should the scoring system of level reviews be abolished on here?

Post by rufierto » 18 Jun 2018 17:07

We are still here having this argument? Listen, I have had my issues with the system, for those that remember, and there may be a better system. But it would not work for this site. This site has been here 18 years someone said? There is a community of people willing to play the games the builders upload, and give us feedback on our games, which you are not going to find on any other site. I have put my bruised ego aside more than once here, and moved on, because you are really not going to find a better place, to get your level played, commented on and reviewed(good or bad). I have taken the feedback and tried to make myself the best builder I can, and if all I am is an 8.5 builder, that is what I am. All builders draw their own little crowd, (some littler than others), but that is ok. We are keeping this game we love going, and most of us are building games better than the original ones we paid for. What else could we want. I know it is weird , me trying to be a uniter instead of divider in this category, but we are lucky to have this place to do our thing. :2thumbsup:
Rescue Dutchy!!!!

User avatar
Jorge22
Master Chatter
Master Chatter
Posts: 9439
Joined: 21 May 2003 03:26
Location: Portugal (Porto area)

Re: Should the scoring system of level reviews be abolished on here?

Post by Jorge22 » 18 Jun 2018 17:07

You know, if you give a test, for example, and it has loads of questions or yet another that, even though it covers all the subjects, is much shorter, students will go all tense and nervous with the big ones. Little do they know that a bigger test does favour their results. My tests (when I'm actually teaching kids as currently I've been working with adults and it's much worse - a few sentences, one instruction and, please, do write an essay - only, they're adults) are actually of the second kind and the kids go "wow, this is great, it's not huge at all". Well, but the thing is they have to be aware of everything. When they fall, they fall from much higher and lose comparatively a lot more. I prefer it that way, anyway; they shouldn't study for a test the day before (if at all). Those who follow me do have no issues and the test is shorter, giving them more time to think.

Where am I getting at with this?

Rating four categories from 0 to 10 actually seems to be fairer than the option. I can't think of anything more unfair than rating between 0 and 1 (I know that was a hyperbole). It's unfair both on the reviewers and on the builders and eventually on those who try to pick a level to play.

Or... in the end the result may amount to the very same, which is also a possibility.

Yes, mistakes will happen in any each case. But less choice may be worse. If that matters at all because there are also people who, in their own right, say it's all in good clean fun and what do you care.

____________________________________________________

Just a slightly off-topic add-on: there's also a trend in education to make everybody cooperate in a way that kids will get exactly the same test at the same time. I think it's stupid because they're all different (as are the teachers and everything else). True, I'm away in the evening every day of the week but if I can choose I prefer teaching adults and discussing things with adults and (even though "regular" classes are over by now and it's exams time and mine aren't) feeling I'm actually much freer with adults until I'm finally able to retire.
Image

User avatar
Tolle87
Member
Member
Posts: 947
Joined: 12 Oct 2012 10:31
Location: Belgrade, Serbia
Contact:

Re: Should the scoring system of level reviews be abolished on here?

Post by Tolle87 » 18 Jun 2018 18:04

Jorge22 wrote:
18 Jun 2018 17:07


Rating four categories from 0 to 10 actually seems to be fairer than the option. I can't think of anything more unfair than rating between 0 and 1 (I know that was a hyperbole). It's unfair both on the reviewers and on the builders and eventually on those who try to pick a level to play.
I fully agree with that part. I might add that I don't think Dj's proposition was a hyperbole because I already talked with him about that long ago and he proposed a simple yes/no (or 0/1) reviews. Obviously, there would be a problem because 7ish and 10ish levels would be treated equally.

The only "real" flaw that I see in current system is that there are three non-gameplay categories, compared to one gameplay category. Gameplay is and forever will be the most important part of any game. What might bring some balance is to divide gameplay into 3 sub-categories (puzzles, physical tasks, creativity) so we would have a total of 6 categories, equally divided into gameplay-based and visual-based ones. However, some of you asked a very legitimate and logical question - why changing something that has been working for 18 years?

Well, here is my reason:

You mustn't forget that when the editor came out, there were only basic wads. No custom objects, no spectacular textures, very limited soundtrack and very VERY tight limits of the editor itself. Getting good marks was really difficult but most importantly - equal for all categories, divided only by personal talents and priorities of the builder.

Today, we have tons and tons of objects, textures, higher object limits....which means it's more easier to get 9's and 10's for visual categories. You don't have to place objects as thoughtfully as you had before, it's enough to download beautiful objects and to place them rationally to get 8+. Although I generally agree with Michael's "don't fix what's not broken" principle, I think the original system IS broken because of this main point - All these changes affected three categories, BUT the gameplay category remained the same. THAT is the problem. Visual aspects are now really favored compared to gameplay, both practically (easier to get good marks with modern editor, objects, textures, audio tunes...) and mathematically (3/4 non-gameplay categories). While it now offers way more possibilities for visuals compared to the original state (which the rating system was originally based on), creating good gameplay still requires as much work and thought as it did before to get good marks. In my opinion, that difference between now and 18yrs ago which favors non-gameplay categories, while still leaving gameplay stand where it was before, should be the reason for some adjustments to compensate that.
A man who introduced me to Lara back in 1996, more than a friend, my second father....
Dusan "Dule" Petrovic (1965-2018)

Marksdad
Mod
Mod
Posts: 4839
Joined: 05 Oct 2001 16:38
Location: UK

Re: Should the scoring system of level reviews be abolished on here?

Post by Marksdad » 18 Jun 2018 22:59

A very interesting and valid point,Tolle.
I agree with your points;but the problem with implementing an updated system is that you'll end up with (to all intents and purposes) two level indexes within the same one. i.e.the old reviews;and the post-revision reviews.
I think we're just going to have to accept that the imbalance is a thing,and take that into account for most new releases.

Now that I'm on this subject of review weaknesses:
Something I've noticed over time is that certain types of levels achieve high scores by dint of appealing only to players with expert playing skills. To give an example:
A while ago,I downloaded a level and began to play it;but after 15 minutes or so,I came to the conclusion that it was aimed squarely at the most expert of players.I had no inclination of spending several days performing tough-as-nails pixel precise banana jumps etc,and binned it.Obviously, because I didn't get very far,a review was out of the question.
Nonetheless,the level ended up with comparatively few,but extremely high,ratings;because the only people completing,and reviewing it,were bound to love it! Everyone else just gave it a miss,and were therefore unable to review it and express their opinions through the rating system - so it consequently sneaked into the upper reaches of the ratings almost by default.
Not that there is anything to be done about this. Similar to the unfair advantages that Tolle pointed out,it simply is what it is - a quirk of the system.

Phil
Mod
Mod
Posts: 5167
Joined: 12 Apr 2002 22:57
Location: Charlotte, NC, USA

Re: Should the scoring system of level reviews be abolished on here?

Post by Phil » 18 Jun 2018 23:50

Marksdad wrote:
18 Jun 2018 22:59
Not that there is anything to be done about this.
Maybe there is. On a number of occasions José has reviewed a level he was unable to complete for one reason or another. I know the practice is generally frowned upon, and I recall that Michael once rejected my review of a highly regarded TR2 level because I could never get past the game crashes (either on my home computer or my office computer) and simply wanted to register the problem. But as long as the reviewer admits he was unable to complete the level, and explains why, I feel that such reviews should be accepted and would provide a better-rounded evaluation of the level from reviewers of all skill levels.

User avatar
Jorge22
Master Chatter
Master Chatter
Posts: 9439
Joined: 21 May 2003 03:26
Location: Portugal (Porto area)

Re: Should the scoring system of level reviews be abolished on here?

Post by Jorge22 » 19 Jun 2018 08:46

I think I may have done that myself a couple of times. Generally speaking though, if I can't complete it I don't review it.
Image

User avatar
Teone
Member
Member
Posts: 715
Joined: 16 Mar 2017 15:59
Location: Italy

Re: Should the scoring system of level reviews be abolished on here?

Post by Teone » 20 Jun 2018 13:10

Marksdad wrote:
18 Jun 2018 22:59
Nonetheless,the level ended up with comparatively few,but extremely high,ratings;because the only people completing,and reviewing it,were bound to love it!
Yes, one of the bigger problems is statistical: more reviews, more accurate is the average score. I rarely happened to disagree with a rating that is the result of many reviews, while if the reviews are few it may happen to disagree.

As newbie who started playing custom levels since one year (more or less), I think the current ranking system is good. In the last year I choosed the games to play sorting the search list by rating, but sometime also sorting by numbers of download.

Insted if you sort the list by likes, you find out a curious thing: the most liked game currently is a Maksdad game rated only 6.29. So probably the like/dislike system would not be so accurate.

User avatar
Jorge22
Master Chatter
Master Chatter
Posts: 9439
Joined: 21 May 2003 03:26
Location: Portugal (Porto area)

Re: Should the scoring system of level reviews be abolished on here?

Post by Jorge22 » 20 Jun 2018 13:54

Agreed. To me the like/dislike feature is sort of useless. I don't even look.
Image

Marksdad
Mod
Mod
Posts: 4839
Joined: 05 Oct 2001 16:38
Location: UK

Re: Should the scoring system of level reviews be abolished on here?

Post by Marksdad » 20 Jun 2018 13:57

:blush: Put me in a corner,there!
Ok,here's the story behind that..
my Son (the builder VonCory) did a school project some years ago (he's a science post-grad now) where he was tasked with creating an algorithm.
He developed one for automatically increasing the hit rates on Internet sites. He was a member here,and chose one of my levels because he felt that no-one would possibly notice :-o
Hence 1000300 likes! (Mists of Avalon has a paltry 435383 by comparison)
Just ignore and move on :)

Btw,he's since moved to Australia;which I feel is punishment enough :hammer: (Joke!!)

Marksdad
Mod
Mod
Posts: 4839
Joined: 05 Oct 2001 16:38
Location: UK

Re: Should the scoring system of level reviews be abolished on here?

Post by Marksdad » 20 Jun 2018 13:59

To me the like/dislike feature is sort of useless. I don't even look.
Well,VonCory's comparatively easy manipulation of it would seem to back that up

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest