Site scoring system (Feb 2nd 2013)

Information of the Rating system.

Re: New site scoring system (Feb 2nd 2013)

Postby Bob.Durf » 18 Jan 2013, 18:57

Looks like a more accurate ELO system (like in Chess)...pretty cool!
Cheater-Hunter (Fired on account of incompetence according to top secret reports)

There's a saying amongst the moderators: "If a job's worth doing, it's worth doing well. If it's not worth doing, give it to Bob." Promotion prospects: Comical
User avatar
Bob.Durf
 
Posts: 2442
Joined: 04 Jan 2009, 02:05
Location: South Carolina
Class: Diplomat
Standard rating: (970)
All-game rating: (969)
Timezone: GMT-5

Re: New site scoring system (Feb 2nd 2013)

Postby super_dipsy » 18 Jan 2013, 21:13

Cspeiker, i shd have said - u will still be able to show your variant rating just like today. So for example I can show my Chaos or AoE or whatever rating just as u do you scores today
User avatar
super_dipsy
Premium Member
 
Posts: 12068
Joined: 04 Nov 2009, 17:43
Class: Ambassador
Standard rating: 1000
All-game rating: 931
Timezone: GMT

Re: New site scoring system (Feb 2nd 2013)

Postby super_dipsy » 18 Jan 2013, 21:36

The feedback has been very useful - I am going to update the FAQs to cover some of these additional points.
User avatar
super_dipsy
Premium Member
 
Posts: 12068
Joined: 04 Nov 2009, 17:43
Class: Ambassador
Standard rating: 1000
All-game rating: 931
Timezone: GMT

Re: New site scoring system (Feb 2nd 2013)

Postby BlueBorjigin » 18 Jan 2013, 21:57

super_dipsy wrote:16. Will you be publishing the algorithm the site uses to calculate ratings adjustments?
• No. For many people it would just be confusing, and for others it would make it easier to play the system rather than the game in the same way that people learnt how to play the old system. it would also be guaranteed to cause far more arguments then it avoids. In short, there is nothing to gain and a lot to lose from making it public.

This raises concerns for me. Players need some indication of how entering a game will affect their ranking. If the algorithm isn't released, then an equally-good or better alternative would be for the site to show you how winning (either by draw or by solo) a certain game could affect your score. This is an issue because of the fact that draws can lower your score. If in some games you need to solo to come out positive, then that's a very high demand, and players should have warning of this. Perhaps this idea can be implemented in a way that tells you when a game has a high risk, but prevents people from abusing it?

Also, if draws can lower your score, can solos lower your score?

I've also got to mention - this isn't a ploy to get more Premiums, is it? There's a great advantage in Anon games now, to prevent headhunting. ;D
User avatar
BlueBorjigin
 
Posts: 222
Joined: 21 Oct 2011, 21:28
Location: Toronto
Class: Star Ambassador
Standard rating: (1160)
All-game rating: (1177)
Timezone: GMT-5

possible to rank by Gunboat-only games?

Postby JohnFletch » 18 Jan 2013, 22:06

I play a lot of tactical Gunboat games. Very fun, and not as time-consuming as normal Diplomacy (though I make sure to always be in at least one of those!) But, I don't see a way to see rankings using a filter of Gumboat. Wouldn't it be easy to rank people based on Gunboat-only games?
User avatar
JohnFletch
Premium Member
 
Posts: 43
Joined: 07 Aug 2011, 22:32
Location: Charlottesville, Virginia
Class: Ambassador
Standard rating: 1069
All-game rating: 2144
Timezone: GMT-5

Re: New site scoring system (Feb 2nd 2013)

Postby Piquinya » 18 Jan 2013, 22:11

BlueBorjigin wrote:
I've also got to mention - this isn't a ploy to get more Premiums, is it? There's a great advantage in Anon games now, to prevent headhunting. ;D


I think ranked games should be anonymous to prevent headhunting. This is one of the details that should be changed to make the new system even more accurate. But, IMHO, the new system is definitely a big step forward in the right direction! :D
Former Moderator of the Oldies Group.
Member of the Classicists, Platinum tier.

Quo usque tandem abutere, Catilina, patientia nostra?
User avatar
Piquinya
 
Posts: 471
Joined: 23 Jul 2010, 16:51
Location: Venezuela
Class: Star Ambassador
Standard rating: (1429)
All-game rating: (1569)
Timezone: GMT-4

Re: New site scoring system (Feb 2nd 2013)

Postby super_dipsy » 18 Jan 2013, 22:32

JohnFletch wrote:I play a lot of tactical Gunboat games. Very fun, and not as time-consuming as normal Diplomacy (though I make sure to always be in at least one of those!) But, I don't see a way to see rankings using a filter of Gumboat. Wouldn't it be easy to rank people based on Gunboat-only games?

ah - yes this is a tricky one. I have had a think about it on a number of occasions, but I keep running into the complexity of how to do it without opening pandora's box.

Once again, this is nothing to do with the new system, which is exactly the same as the old system. But here is the issue that makes it tricky.
At the moment, you can show game results and scoring (will be rating in the new system), and you can select by 'variant'. So you can have All, Classic, Fleet Rome, Chaos, Build Anywhere, AoE. In the new system you can also see the TrueDip rating. Now could we add Gunboat? Yes, of course - but of course gunboat is an option that can apply to All, Classic, Fleet Rome, .... So do we show Gunboat for each variant? Or just a total value? And then what about FoW and Stuff which can ALSO apply to all the variants. And then what about the different map variants, which apply to all variants and also to PPO, Gunboat, Stuff, FoW, .....

Essentially, the way to do it would be to have something akin to what we have when you do a game search, where you select the exact combination of values you want. BUT this now means we have to keep a score record for each variant/map/game type combination, which starts to mount up exponentially.

Not a very satisfactory answer, and something that does trouble me, but I honestly am not sure it is worth the investment to sort it. And since it is not a feature in the current system and does not seem to have been desperately missed...
User avatar
super_dipsy
Premium Member
 
Posts: 12068
Joined: 04 Nov 2009, 17:43
Class: Ambassador
Standard rating: 1000
All-game rating: 931
Timezone: GMT

Re: New site scoring system (Feb 2nd 2013)

Postby duckling » 19 Jan 2013, 01:38

super_dipsy wrote: 16. Will you be publishing the algorithm the site uses to calculate ratings adjustments?
• No. For many people it would just be confusing, and for others it would make it easier to play the system rather than the game in the same way that people learnt how to play the old system. it would also be guaranteed to cause far more arguments then it avoids. In short, there is nothing to gain and a lot to lose from making it public.


I really like the sound of the new system. But this part bugs me a bit. Doesn't this mean that playing the system is now only possible for those who are able to infer how it works, and you're not giving the rest of the players a chance to detect them (since they can't tell)?

If it's that easy to play the system, that you expect visible problems if the algo is published, it doesn't really inspire confidence in the idea that it will be hard to figure out (well enough) anyway, for a smart, frequent player.

Or am I too much of a pessimist?
duckling
 
Posts: 142
Joined: 06 Jan 2010, 17:48
Class: Star Ambassador
Standard rating: (1130)
All-game rating: (1176)
Timezone: GMT+1

Re: New site scoring system (Feb 2nd 2013)

Postby JohnFletch » 19 Jan 2013, 07:45

super_dipsy wrote:
JohnFletch wrote:I play a lot of tactical Gunboat games. Very fun, and not as time-consuming as normal Diplomacy (though I make sure to always be in at least one of those!) But, I don't see a way to see rankings using a filter of Gumboat. Wouldn't it be easy to rank people based on Gunboat-only games?

ah - yes this is a tricky one. I have had a think about it on a number of occasions, but I keep running into the complexity of how to do it without opening pandora's box.

Once again, this is nothing to do with the new system, which is exactly the same as the old system. But here is the issue that makes it tricky.
At the moment, you can show game results and scoring (will be rating in the new system), and you can select by 'variant'. So you can have All, Classic, Fleet Rome, Chaos, Build Anywhere, AoE. In the new system you can also see the TrueDip rating. Now could we add Gunboat? Yes, of course - but of course gunboat is an option that can apply to All, Classic, Fleet Rome, .... So do we show Gunboat for each variant? Or just a total value? And then what about FoW and Stuff which can ALSO apply to all the variants. And then what about the different map variants, which apply to all variants and also to PPO, Gunboat, Stuff, FoW, .....

Essentially, the way to do it would be to have something akin to what we have when you do a game search, where you select the exact combination of values you want. BUT this now means we have to keep a score record for each variant/map/game type combination, which starts to mount up exponentially.

Not a very satisfactory answer, and something that does trouble me, but I honestly am not sure it is worth the investment to sort it. And since it is not a feature in the current system and does not seem to have been desperately missed...


Right, sorry it's in the new system thread. You could - if you did it- track only the very most popular combinations. As for me, I'd vote for just the Gunboat flag - on or off.
User avatar
JohnFletch
Premium Member
 
Posts: 43
Joined: 07 Aug 2011, 22:32
Location: Charlottesville, Virginia
Class: Ambassador
Standard rating: 1069
All-game rating: 2144
Timezone: GMT-5

Re: New site scoring system (Feb 2nd 2013)

Postby super_dipsy » 19 Jan 2013, 08:23

duckling wrote:
super_dipsy wrote: 16. Will you be publishing the algorithm the site uses to calculate ratings adjustments?
• No. For many people it would just be confusing, and for others it would make it easier to play the system rather than the game in the same way that people learnt how to play the old system. it would also be guaranteed to cause far more arguments then it avoids. In short, there is nothing to gain and a lot to lose from making it public.


I really like the sound of the new system. But this part bugs me a bit. Doesn't this mean that playing the system is now only possible for those who are able to infer how it works, and you're not giving the rest of the players a chance to detect them (since they can't tell)?

If it's that easy to play the system, that you expect visible problems if the algo is published, it doesn't really inspire confidence in the idea that it will be hard to figure out (well enough) anyway, for a smart, frequent player.

Or am I too much of a pessimist?

I think this may be a storm in a teacup :) . Do you play Chess? I was a chess champion when I used to play regularly as a kid, and I knew what my elo rating was. I also knew it went up or down when I played a game, and that if I beat someone with a higher rating than me it went up more, but when I lost to someone lower than me it hurt more. I did not know (and still do not) what formula elo used. I know the principles of it, but if you said to me what would your rating be changed to if you beat a player with a rating of 2000 I would not be able to tell you. I do not believe many chess players WOULD be able to tell. Similarly, now that I play most of my chess online I also have a site rating for chess, but I do not know how it is worked out. I simply know how it will BEHAVE. I also have onsite ratings for backgammon and other games I play - similar story, they behave like elo systems but I do not know the formula. And do you know, I am glad I don't know. If I did, I might start saying 'there is no point in me playing that person if I want to get to 2000, because the risk to me is too great and I know I will only get +3 for beating him'. Instead, I go into such a game thinking 'I know I am stronger in rating terms, so I better make sure I win'.

Part of the problem with the old system is that because it was so clearly specified, people were able to play to the specifics of the system with much less focus on the game. People knew for instance that they could join a surrendered game and if they do not like what they see they can surrender immediately - +1 / -1. As long as it did not push a good score off your fading echoes, no harm no foul. People also knew that if you watch for players getting kicked out of games for cheating just before they win, you can pick up an almost immediate solo for no investment. This is nothing to do with playing Diplomacy, it is about playing the system.

The long list of characteristics in the OP of this thread explains HOW the rating system will affect you as you play your games. Just as the Chess elo rating system does. It does not tell you exactly how it is worked out, but as I said in the chess example I am of the opinion hardly any chess players could tell you anyway. But the rating system works just fine, and I believe ours will too.

So there is little to be gained and more to lose by publishing the algorithms. The only gain you have come up with, which in my mind is pretty tenuous, is that if a maths geek were to study the games and results they might be able to infer the algorithm and somehow get an advantage from that. I would be fascinated to see that in action - particularly since in order to work out the results from each game in rating terms the person would have to take at least daily or intraday snapshots of the ratings table to see what change each player had for each game. This is something individual players can see, but seeing someone else's information is much harder. But even if someone did reverse engineer the algorithm, what is the benefit they could achieve? The most, it seems to me, would be the ability to say 'to reach the new position on the leaderboard I need exactly 17 more points, and I know that I can work out that if I play 6 other players with ratings totalling x and I get a 3 way draw I will get 17'. So how does that help? How can I control who enters my game? OK, I could make it pw protected - but what if someone surrenders? Now anyone can join - what does that do to my result? And (as I said eslewhere) the result is worked out on ratings at the point of finish anyway, so knowing a player's rating at the start of the game does not necessarily guarantee me the 17 points I had planned. does this really sound like a big benefit?

The final reason for not publishing the algorithms is that I guarantee that whatever algorithm we published, there would then be a never-ending round of criticism from people who don't like it, or want a different one, or whatever. And I will also guarantee that there would never be any consensus. If you want some evidence of this, take a quick scan of any one of the multitude of scoring system debates on suggestions - warning though, they can easily be 20 pages long, so you have a lot of reading ;).

Does this help? I am not trying to be difficult - just trying to do what I personally feel is best for the site. I could easily be wrong, and have been many times before. But I do not know any other way to progress something when you have no sense of consensus. The alternative is nothing ever happens :( .
User avatar
super_dipsy
Premium Member
 
Posts: 12068
Joined: 04 Nov 2009, 17:43
Class: Ambassador
Standard rating: 1000
All-game rating: 931
Timezone: GMT

PreviousNext

Return to Site Scoring System

Who is online

Users browsing this forum: No registered users and 2 guests