Site scoring system (Feb 2nd 2013)

Information of the Rating system.

Site scoring system (Feb 2nd 2013)

Postby super_dipsy » 18 Jan 2013, 07:13

New rating system
The most fundamental change of the new rating system compared to the old one is that your score is not the total of your game results calculated at a flat rate (eg solo=12, 2-way draw=6 etc), but instead is a rating that attempts to reflect your playing strength. Your rating is adjusted for each game you play based on how strong or weak your opponents were and what result you were able to achieve. Getting a good result against players with higher ratings will give your rating a greater lift than when achieved against players with lower ratings. Also, unlike today, your rating can go down as well as up. So for example when you lose a game your rating will be adjusted down, although once again if you lose against a strong group of players your rating will suffer less than if you lose against a weaker set of players. The idea is that your rating will stabilize around your current playing strength, gradually increasing as you become a better player.

There are a few important characteristics of our new PlayDip rating system that are worth summarizing:
• A solo win has been weighted more heavily than draws (which themselves are graduated) and is never < 0
• A game result can make your rating go down as well as up (see Notes)
• Ancient Med games score proportionally less than classic ones because there are less players
• There is no ‘fading echoes’ any more - ratings reflect your playing level, not total score
• All players (and all new players joining later) start with a rating of 1000
• The system tries to find your level as fast as possible, so for your first 10-20 games your rating may jump about a lot as the system tries to find the appropriate rating for you. As you play more games, the size of fluctuations will gradually level off
• If you join a surrendered game, your result is based on the % of turns you played the country (until either it is eliminated or the game finishes) EXCEPT ....
• ...ALL surrenders, whether in a game you started or joined as a replacement, score a 100% loss, regardless of how many turns you played the country
• A ratings ‘shield’ has been introduced; when active, this protects your rating from any negative result when you take over a surrendered position (see Notes)
• The site will track two ratings for each player – a Site rating and a PureDip rating (see Notes)

Notes
Some of these points merit a little further detail. While it is impossible to give a fixed number of how much a Solo is worth compared to a 2-way, 3-way etc, since it will depend on the ratings of the other players, the Solo adjustment in the new system has been deliberately weighted so that it is worth substantially more than two ‘2-way’ draws. Note also that since a draw depends on the number of sharers and relative player ratings in the game, it is not guaranteed to generate a positive change to your rating although in most cases it will.

The ratings ‘shield’ is intended to encourage players to pick up some surrendered positions and play them out. This is a service to the community that is very much appreciated, since everyone would prefer to play games with all the countries involved, but often the only games that get picked up are when the surrender occurs in a strong position because players do not want to risk their own ratings. The ratings shield, when active, gives you a ‘free play’, protecting your rating from any negative result but allowing you to still take advantage of anything you do manage to achieve. You can therefore take up a surrendered position without any risk to yourself (and maybe even get more points!).

The way the shield works is that to be used it has to be charged. Your shield is charged by beginning (not just joining but starting) a ranked game, and the charge will last until it is activated by joining a surrendered game. Only one use per charge is allowed, so once you have used it you will need to begin another ranked game to recharge it. If your shield is active, your rating will not go down when the game finishes, unless you surrender too which negates the shield benefit. Note that you cannot 'save up charges - if you start 5 ranked games in a row, it still only has enough charge to protect you in ONE game before you then have to recharge it. Hopefully, the shield will encourage players to take up surrendered positions just to help out others in the secure knowledge that nothing bad can happen. But the ‘alternating’ nature of having to begin a ranked game to charge it each time before it can be used prevents players from never risking their ratings by only picking up surrendered positions.

Apart from the shield, remember that scores are adjusted if you did not play the country throughout the game. If you join a game on turn 11 and play for ten turns before getting a draw, your result will be calculated at 50%. However, if you surrender then there is no percentage reduction – you score a full loss as if you had played the entire game.

With the new system, your Site rating determines your position in the site rankings. However, there will be another rating for all players on the site; a PureDip rating. Only ranked games count towards ratings, but the Site rating will include your results in ALL ranked games, no matter what options and variations. However, your PureDip rating is an attempt to provide a rating for playing what some people might call ‘full’ Diplomacy – that is, full negotiations, normal rules, etc.. The following is the list of games that count to the PureDip rating:
• All maps
• Only games with full negotiations (no PPO, no Gunboat)
• No rule variations (no Fog, Stuff, Build Anwhere, Chaos)
• No unit variations (no Winter 1900, Age of Empires, Fleet Rome)
• Public, private and anonymous games all included

FAQ
1. Why are we changing the scoring/rating system?
• The current system had the great advantage of being simple. However, most game sites have rating systems in place that take account of the strength of the opposition, meaning that if you do well against a stronger player you gain more than against a weaker player. We have also had a lot of requests from players to have such a system. And with this being our 6th year, we thought it was time to give everyone a chance to start from scratch again!

2. How much is a win/draw/loss worth?
• Because the new system bases changes to your rating in each game based on not only your result but the strength of the opposing players, there is no fixed value for a win, draw or loss. Just remember that a solo will score much better than a draw, draws score better when shared between fewer people and the ratings shield will protect you from any loss if it is active. Remember also the fundamental basis of the new system - if you play players on average weaker than yourself you will score less for a win and be hit harder by a loss; and if you play others generally stronger than yourself you will score more for a win and be hit less by a loss.

3. I got a draw, but my rating went down. How can that happen?
• Points for draws scale down from 2-ways, but the scale depends on the ratings of the other players. So for example, if you are much higher than the others, a 4-way draw may well lose you points because you were expected to do better given your rating.

4. What’s a good rating?
• Ratings in the 1500-2000 region can be considered to be very respectable. Ratings of over 2000 are likely to put you clearly in the top level of players on the site

5. You say my result is based on the ratings of the opposition. But what if a country has been played by more than one person?
• It may help to think of your result points being calculated based on the ratings of the COUNTRIES you played against. In normal cases, that is the rating of the player who played the country. But if a country was played by more than one person (someone surrendered and a replacement came in) then that country's rating as far as the scoring is concerned is the average of the ratings of the different players over how many turns they played. So for example, if France was played by a 1200 rated player for 4 turns and then a 800 rated player for 6 turns, the country rating for result calculations would be (1200*4 + 800*6)/10 = 960.

6. Why have you weighted solos more heavily than draws?
• The intention of Diplomacy is that you play to conquer all your rivals, to be the sole ruler of the universe. Online games, however, can go on for a long time, and players may lose interest or be prepared to agree a draw. We therefore support draws, and offer credit for achieving them, but a solo is still the blue ribbon of Diplomacy, and should be recognized as such. A solo should NOT be equivalent to a ‘double 2-way’ – it is much better than that!

7. What if I lose my first few games – can I ever recover from a really low rating?
• Absolutely! One of the big advantages of the new system is that as you improve, your rating can climb quite rapidly. One solo can surge you the equivalent of 5-10 losses, and even a draw against better players will give you a good lift. This is particularly good for new people coming to the site, who may well struggle initially but once they pick up the game they can advance strongly

8. Why have you got rid of fading echoes?
• The purpose of fading echoes (where older results count less) was twofold; to enable players who started badly to eventually discard those results, and to prevent players getting to the top of the leaderboard simply by playing vastly more games. In other words, it was solving issues created by having a cumulative scoring system. The new system is designed to match your playing level. A run of good results can more than cancel out a run of bad ones, and also as player ratings get higher and higher it gets harder and harder to generate big swings since they are more likely to be playing lower rated players.

9. Why have you taken away the -1 on Surrenders? We don’t like surrenders!
• True, there is no flat rate penalty for a surrender. But remember, in the new system a loss does not mean 0 points – instead a loss means a negative adjustment to your rating. This is irrespective of how many turns you play before surrendering. As for penalizing surrenders more than other losses, this is something we are looking at dealing with outside of the rating system. Surrendering does discharge your shield though.

10. I play a lot of PlayDip games, so my rating will always be changing. How do you determine what rating to use to calculate me score when I finish a game?
• For both this reason and also to maintain scoring consistency, the rating used to determine your result in any game is your rating at the point in time the game ends (or you surrender), NOT when you started it. In other words, every game result is determined chronologically.

11. Why is the ratings shield not available on ALL games where you replace a surrendered country?
• The shield is to encourage players to pick up positions that would otherwise be unfulfilled as well as good positions. The ideal would be every player would give a bit back to the community by picking up a proportion of surrendered positions, but if protection was for all replacement games, people could hide behind the shield and never have to risk their rating, by playing nothing but surrendered positions.

12. How do I know if my ratings shield is charged / active?
• When you go to the Join Game screen, there is a message on the top right that tells you if your shield is Charged or Drained. If it is charged and you join a surrendered (ranked) game, then from now on your Active Games list will include a (Ratings Shield Active) indicator next to the stats = rank field in the game description. So you will always know which games in your Active list are ratings-protected.

13. Tournament Diplomacy scoring often takes into account your game position in the scoring, such as number of supply centres and/or territories controlled. Why does the new system ignore this?
• Remember, the aim of Diplomacy is to win. Draws are a way of agreeing to close the game down for convenience, with all players sharing the draw receiving the same. Tournaments have to sort ranks out between players over an extremely small number of games, and so it is vital to have more differentiation in the scoring. This is why they take into account other factors. We have the luxury of being able to build a rating from a larger number of games, so we do not need to bend the scoring system in this way.

14. I worked hard to get where I am in the rankings. Why not start people with a rating that reflects their previous score?
• Part of the reason is to give everyone a new challenge, and a chance to prove (or re-prove) themselves. However, a major reason is that the new scoring system is designed based on different principles, and it may seem just as unfair to others that players can carry over scores from a different system. With the new system, for example, there is much less to be gained from playing against a bunch of noobs or only picking up strong, surrendered positions. The new system also rewards solos and wins against strong players more heavily. I am confident that the strong Diplomacy players will have no trouble reasserting their positions in the new rankings, though, particularly since unlike the old system where you were never really going to catch up until you had played 50 games and filled your 'fading echoes' slots, in the new one you wont need nearly as many games!

15. How accurate is the new system?
• The new system does not pretend to be a precise rating of you as a Diplomacy player. However, it does at least produce a rating for you after each game that takes into account your result compared to the ratings of the people you played. On that basis, it is a reasonable reflection of how you compare to other players.

16. Will you be publishing the algorithm the site uses to calculate ratings adjustments?
• No. For many people it would just be confusing, and for others it would make it easier to play the system rather than the game in the same way that people learnt how to play the old system. it would also be guaranteed to cause far more arguments then it avoids. In short, there is nothing to gain and a lot to lose from making it public.

17. What will happen to games in progress when the new system starts? And what will happen to my old score?
• All games that START (not created or confirming, but actually start) after the announced switchover time will be scored on the new rating system. However, games in progress at that time will still use the old scoring system. Even if you join a game after the switchover to replace a surrendered country, if the game started BEFORE the switchover then you will be scored in that game on the old system. Once all these games finish, the old system will be retired. But the old scores will be retained in a Hall of Fame for posterity. The Hall of Fame will retain TWO lists - the final table of active players, but also the full table of ALL players (including those who may have left and were dropped from the table as inactive)

18. Will player records (eg wins, losses, types of games played, surrenders etc) be kept under the Statistics tab?
• At the moment there are no plans to change this. Player statistics (only available to premiums) will show the player's rating rather than score, of course, but the game statistics will be maintained with the full history. It is possible later that we may split the statistics into 'old system' and 'new system' ones, but not until more data has been assembled.

19. What will show on the Statistics/Points tab after Feb 2nd when the new system comes in?
• After the switchover, this section will contain two parts, the new system results and the old ones. So you will see your rating/ranking and a list of your game results under the new system, followed by the same display as you see today listing your score (for games that were started before 2nd Feb) and game results in the old system. Once the old system is complete (no more unfinished games that started before 2nd Feb) we can either leave the information for posterity on the display or remove it. Note also that just as today with your score and game history, you can ask for your rating/game history by variant (eg Chaos, AoE etc) or the TrueDip information.

20. While there are still games playing that started before 2nd Feb, will you still show scores and the old rankings?
• No, apart from as covered in Q19. The rankings list on the home page will be ratings-based not old score-based, and the entry on your forum block will give your rating-based rank too. However, don't forget that all old-system games will still count to the final rankings / scores list that will be archived in the Hall of Fame when all old games are complete.

21. I notice that on the forum where it used to give my ranking, it now gives my rating. Why the change?
• There will always be an interest in trying to get into the top 10, or even the top 100, but on most game systems people focus instead on their own ratings. Even if you are ranked 3000th, you still want to try to increase your rating. In the old system it would have made little sense to show a total score because this was cumulative anyway. But in the new system, we can now show your rating much the same as other game sites typically do.

22. Why did I not get 400 for my solo when I see others got 400?
• There is a post to cover all sorts of questions about why rating results might differ. You can find it here http://www.playdiplomacy.com/forum/viewtopic.php?f=129&t=35683

23. What happens if a finished ranked game has to be unranked eg for cheating?
• In the situation where a ranked game has to be unranked, any adjustments made to the rankings of the players involved in the game will be reversed. So for example, if a player won the game and got a ranking increase of 50, that player will receive a rating reduction of 50 at the point the game is unranked.
User avatar
super_dipsy
Premium Member
 
Posts: 12068
Joined: 04 Nov 2009, 17:43
Class: Ambassador
Standard rating: 1000
All-game rating: 931
Timezone: GMT

Re: New site scoring system (Feb 2nd 2013)

Postby flowergreg » 18 Jan 2013, 11:54

Will it be possible to create games only for high-ranked players? Having a lot of low-ranked opponents may be risky...
flowergreg
 
Posts: 5
Joined: 16 Jan 2013, 15:16
Class: Ambassador
Standard rating: (985)
All-game rating: (963)
Timezone: GMT

Re: New site scoring system (Feb 2nd 2013)

Postby Pedros » 18 Jan 2013, 12:12

Usual system - password and advertise them in the Games Forum.

At least that's one improvement in this system - under the old one it was playing decent opponents that was dangerous; playing poor ones sent you way up the chart!! :mrgreen: :mrgreen:
"Sooner or later, one of us will stab the other. But for now we're both better off as allies" (kininvie)
User avatar
Pedros
 
Posts: 12465
Joined: 25 Jan 2009, 12:59
Location: Somewhere full of gorse and brambles, West Cornwall
Class: Star Ambassador
Standard rating: (1085)
All-game rating: (1314)
Timezone: GMT

Re: New site scoring system (Feb 2nd 2013)

Postby gareth66 » 18 Jan 2013, 13:07

Can I clarify the reference to 2 February? Presumably this is the implementation date, but does it apply to all games created on or after 2 Feb, all games that start on or after 2 Feb or all games that finish on or after 2 Feb?

Thanks.
User avatar
gareth66
Moderator
Moderator
 
Posts: 3523
Joined: 06 Apr 2011, 18:09
Location: Uk (North Midlands)
Class: Star Ambassador
Standard rating: (1485)
All-game rating: (1638)
Timezone: GMT

Re: New site scoring system (Feb 2nd 2013)

Postby super_dipsy » 18 Jan 2013, 13:13

gareth66 wrote:Can I clarify the reference to 2 February? Presumably this is the implementation date, but does it apply to all games created on or after 2 Feb, all games that start on or after 2 Feb or all games that finish on or after 2 Feb?

Thanks.

super_dipsy wrote:17. What will happen to games in progress when the new system starts? And what will happen to my old score?
• All games that START (not created or confirming, but actually start) after the announced switchover time (Feb 2nd 0600 GMT) will be scored on the new rating system. However, games in progress at that time will still use the old scoring system. Even if you join a game after the switchover to replace a surrendered country, if the game started BEFORE the switchover then you will be scored in that game on the old system. Once all these games finish, the old system will be retired. But the old scores will be retained in a Hall of Fame for posterity. The Hall of Fame will retain TWO lists - the final table of active players, but also the full table of ALL players (including those who may have left and were dropped from the table as inactive)
User avatar
super_dipsy
Premium Member
 
Posts: 12068
Joined: 04 Nov 2009, 17:43
Class: Ambassador
Standard rating: 1000
All-game rating: 931
Timezone: GMT

Re: New site scoring system (Feb 2nd 2013)

Postby EnalanDiplomat » 18 Jan 2013, 14:45

Silly Question - So do we lose all of the points we have earned so far with the site? :roll:
From Chaos Comes Order - Friedrich Nietzsche
From Order Comes Chaos - William James
Faith Manages - Babylon 5 TV show.
EnalanDiplomat
 
Posts: 143
Joined: 16 Aug 2012, 21:21
Location: Peoria Illinois
Class: Diplomat
Standard rating: (1124)
All-game rating: (1103)
Timezone: GMT-6

Re: New site scoring system (Feb 2nd 2013)

Postby super_dipsy » 18 Jan 2013, 15:30

Unfortunately, yes. This is a new start after 5 years of the old system. Everyone will be starting anew from a clean sheet of paper. There are a number of reasons for this, check out the FAQs in the original post. However, the new system does not have a lot of the drawbacks of the old one - for instance, even if you do not play a lot of games you are not disadvantaged by the 50 game slot cumulative issue (where players got the most by ensuring they use all 50 of their fading echoes slots).
User avatar
super_dipsy
Premium Member
 
Posts: 12068
Joined: 04 Nov 2009, 17:43
Class: Ambassador
Standard rating: 1000
All-game rating: 931
Timezone: GMT

Re: New site scoring system (Feb 2nd 2013)

Postby Custer » 18 Jan 2013, 16:13

Are the countries weighted differently? Easier to win with Russia or Turkey than Italy....... :ugeek:
First..........get off my lawn! Second........it's a dashing self portrait! Courtesy of The Craw. Third.....I am still SHIV, Keeper of the Stone Tablets! Go Pack!
User avatar
Custer
Premium Member
 
Posts: 3183
Joined: 24 Jan 2009, 20:29
Location: Sailing somewhere in the Rockies on the Black Pearl but originally a Yooper!
Class: Star Ambassador
Standard rating: 1108
All-game rating: 1071
Timezone: GMT-6

Re: New site scoring system (Feb 2nd 2013)

Postby super_dipsy » 18 Jan 2013, 16:37

No, we still play to the standard Diplomacy rules.

There has however been discussions recently that I have seen (in the Suggestions forum I think) that perhaps the only form of country assignment that should be allowed in Ranked games is random assignment. This is nothing actually to do with the new system - the discussion started on the old one. But it has some merit. If it comes up on Suggestions again we might want to have a poll to see what people think.
User avatar
super_dipsy
Premium Member
 
Posts: 12068
Joined: 04 Nov 2009, 17:43
Class: Ambassador
Standard rating: 1000
All-game rating: 931
Timezone: GMT

Re: New site scoring system (Feb 2nd 2013)

Postby cspieker » 18 Jan 2013, 18:36

Way to shake things up!

I have only a quibble with PPO not being in the "puredip" rating system. That has much negotiation and sometimes must be even more artful than secret negotiations.

I also mourn for all the other variants that might just no longer get played seriously.

Maybe we could have both a "Puredip" and a "Taintedip" rating? Just to keep the fun variants in the mix?

But other than that, I think the surrender shield thing sounds interesting, and might fill NMR holes better. Weighting game worth with respect to opponents is also a wonderful, positive change.
User avatar
cspieker
 
Posts: 469
Joined: 30 Aug 2010, 22:30
Class: Star Ambassador
Standard rating: (1300)
All-game rating: (1381)
Timezone: GMT-8

Next

Return to Site Scoring System

Who is online

Users browsing this forum: No registered users and 1 guest

cron